2024-09-11T08:13:57.406 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T08:13:57.422 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:13:57.504 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7901280' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHP1TUscDPridNQGr0MFuXtlcciw2N1/iKaZMXu2OhYZLzLH5Jd+bjWRLFIwPJiPsEn1A6dXhFCcOLqNB4LJJjI= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6oQWX6PoAMyPZ+Ku80QbVlP6hcK371pBR02NWADLY+fLNnQLkB/i/R97iE0UVSylsAppmwN+OcJ1Y2yJdeM20= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T08:13:57.504 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T08:13:57.505 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T08:13:57.505 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T08:13:57.507 INFO:teuthology.task.internal:Checking packages... 2024-09-11T08:13:57.529 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T08:13:57.530 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T08:13:57.530 INFO:teuthology.packaging:ref: None 2024-09-11T08:13:57.530 INFO:teuthology.packaging:tag: None 2024-09-11T08:13:57.530 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T08:13:57.530 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:13:57.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T08:13:57.678 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T08:13:57.679 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T08:13:57.686 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T08:13:57.686 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T08:13:57.692 INFO:teuthology.task.internal:Saving configuration 2024-09-11T08:13:57.703 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T08:13:57.709 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T08:13:57.735 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 08:08:37.994815', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHP1TUscDPridNQGr0MFuXtlcciw2N1/iKaZMXu2OhYZLzLH5Jd+bjWRLFIwPJiPsEn1A6dXhFCcOLqNB4LJJjI='} 2024-09-11T08:13:57.752 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 08:08:37.993336', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6oQWX6PoAMyPZ+Ku80QbVlP6hcK371pBR02NWADLY+fLNnQLkB/i/R97iE0UVSylsAppmwN+OcJ1Y2yJdeM20='} 2024-09-11T08:13:57.752 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T08:13:57.759 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T08:13:57.759 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T08:13:57.759 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T08:13:57.820 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe08bd73ac0>, signals=[15]) 2024-09-11T08:13:57.820 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T08:13:57.827 INFO:teuthology.task.internal:Opening connections... 2024-09-11T08:13:57.827 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T08:13:57.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:13:57.908 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-09-11T08:13:57.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:13:57.985 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T08:13:57.992 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-11T08:13:58.010 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-11T08:13:58.011 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-11T08:13:58.069 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-09-11T08:13:58.069 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-09-11T08:13:58.069 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-09-11T08:13:58.069 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-09-11T08:13:58.069 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-09-11T08:13:58.069 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-09-11T08:13:58.069 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T08:13:58.069 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-09-11T08:13:58.070 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-09-11T08:13:58.070 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T08:13:58.070 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-09-11T08:13:58.070 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T08:13:58.070 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T08:13:58.070 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T08:13:58.070 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-11T08:13:58.091 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-09-11T08:13:58.109 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-09-11T08:13:58.109 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-09-11T08:13:58.167 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-09-11T08:13:58.167 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-09-11T08:13:58.167 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T08:13:58.168 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T08:13:58.169 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-09-11T08:13:58.191 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T08:13:58.204 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T08:13:58.210 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T08:13:58.211 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-11T08:13:58.213 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-09-11T08:13:58.229 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T08:13:58.236 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T08:13:58.236 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-11T08:13:58.270 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-09-11T08:13:58.293 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T08:13:58.346 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T08:13:58.360 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T08:13:58.361 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T08:13:58.361 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T08:13:58.361 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T08:13:58.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:13:58.378 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-09-11T08:13:58.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:13:58.379 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-11T08:13:58.434 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-503.el9.x86_64 2024-09-11T08:13:58.435 INFO:teuthology.task.kernel:Running kernel on smithi172: 5.14.0-503.el9.x86_64 2024-09-11T08:13:58.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum install -y kernel 2024-09-11T08:13:58.436 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-503.el9.x86_64 2024-09-11T08:13:58.436 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-503.el9.x86_64 2024-09-11T08:13:58.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-09-11T08:14:00.607 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-11T08:14:02.968 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.2 MB 00:02 2024-09-11T08:14:07.685 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 20 MB 00:03 2024-09-11T08:14:09.499 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-11T08:14:11.986 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-11T08:14:13.420 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 19 kB 00:00 2024-09-11T08:14:14.566 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T08:14:15.915 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-11T08:14:17.391 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-11T08:14:18.320 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T08:14:19.954 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T08:14:21.977 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T08:14:21.978 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T08:14:22.081 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:14:22.086 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:14:22.086 INFO:teuthology.orchestra.run.smithi172.stdout: Package Architecture Version Repository Size 2024-09-11T08:14:22.086 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:14:22.086 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout:Install 4 Packages 2024-09-11T08:14:22.087 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:14:22.088 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 79 M 2024-09-11T08:14:22.088 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 126 M 2024-09-11T08:14:22.088 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:14:22.298 INFO:teuthology.orchestra.run.smithi172.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 255 kB/s | 23 kB 00:00 2024-09-11T08:14:22.648 INFO:teuthology.orchestra.run.smithi172.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-11T08:14:23.048 INFO:teuthology.orchestra.run.smithi172.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 41 MB/s | 35 MB 00:00 2024-09-11T08:14:23.515 INFO:teuthology.orchestra.run.smithi172.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-11T08:14:23.516 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:14:23.516 INFO:teuthology.orchestra.run.smithi172.stdout:Total 55 MB/s | 79 MB 00:01 2024-09-11T08:14:24.059 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-11T08:14:24.201 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:14:24.564 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:14:24.564 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:14:25.509 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:14:25.509 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:14:26.141 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T08:14:26.141 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T08:14:26.229 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:14:26.233 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:14:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-11T08:14:26.233 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:14:26.233 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:14:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T08:14:26.233 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:14:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T08:14:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T08:14:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T08:14:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:14:26.234 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:14:26.234 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:14:26.234 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-11T08:14:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:14:26.234 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 79 M 2024-09-11T08:14:26.235 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 126 M 2024-09-11T08:14:26.235 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:14:26.461 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 190 kB/s | 23 kB 00:00 2024-09-11T08:14:28.051 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:14:28.754 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:14:28.841 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:14:31.061 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 7.3 MB/s | 35 MB 00:04 2024-09-11T08:14:31.657 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:14:31.795 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:14:33.044 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:06 2024-09-11T08:14:36.601 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:14:36.720 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:14:39.618 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.2 MB/s | 29 MB 00:13 2024-09-11T08:14:39.619 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:14:39.619 INFO:teuthology.orchestra.run.smithi043.stdout:Total 5.9 MB/s | 79 MB 00:13 2024-09-11T08:14:40.331 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:14:40.700 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:14:40.700 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:14:41.560 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:14:41.674 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:14:41.674 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:14:44.296 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:14:45.111 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:14:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:14:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:14:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:14:53.246 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:14:53.291 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:14:58.424 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:19.883 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:20.274 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:20.274 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:15:20.274 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:15:20.274 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:15:21.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:21.153 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:15:21.154 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:15:21.154 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T08:15:21.154 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T08:15:21.154 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T08:15:21.154 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T08:15:21.154 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:15:21.154 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:15:21.510 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:15:21.588 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:15:21.589 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:15:21.589 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T08:15:21.589 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T08:15:21.589 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T08:15:21.589 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T08:15:21.589 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi172.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T08:15:21.589 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:15:21.647 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:15:21.648 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:15:21.648 DEBUG:teuthology.orchestra.run.smithi172:> sudo rpm -qi grub2-tools 2024-09-11T08:15:21.688 INFO:teuthology.orchestra.run.smithi172.stdout:Name : grub2-tools 2024-09-11T08:15:21.688 INFO:teuthology.orchestra.run.smithi172.stdout:Epoch : 1 2024-09-11T08:15:21.688 INFO:teuthology.orchestra.run.smithi172.stdout:Version : 2.06 2024-09-11T08:15:21.688 INFO:teuthology.orchestra.run.smithi172.stdout:Release : 82.el9 2024-09-11T08:15:21.688 INFO:teuthology.orchestra.run.smithi172.stdout:Architecture: x86_64 2024-09-11T08:15:21.688 INFO:teuthology.orchestra.run.smithi172.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Group : Unspecified 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Size : 8274670 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:License : GPLv3+ 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Packager : builder@centos.org 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Vendor : CentOS 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Summary : Support tools for GRUB. 2024-09-11T08:15:21.689 INFO:teuthology.orchestra.run.smithi172.stdout:Description : 2024-09-11T08:15:21.690 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:15:21.690 INFO:teuthology.orchestra.run.smithi172.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T08:15:21.690 INFO:teuthology.orchestra.run.smithi172.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T08:15:21.690 INFO:teuthology.orchestra.run.smithi172.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T08:15:21.690 INFO:teuthology.orchestra.run.smithi172.stdout:hardware devices. 2024-09-11T08:15:21.690 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:15:21.690 INFO:teuthology.orchestra.run.smithi172.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T08:15:21.691 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T08:15:21.691 INFO:teuthology.task.kernel:Updating grub on smithi172 to boot 5.14.0-505.el9.x86_64 2024-09-11T08:15:21.691 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T08:15:22.326 INFO:teuthology.orchestra.run.smithi172.stderr:Generating grub configuration file ... 2024-09-11T08:15:23.595 INFO:teuthology.orchestra.run.smithi172.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T08:15:23.639 INFO:teuthology.orchestra.run.smithi172.stderr:done 2024-09-11T08:15:23.641 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-09-11T08:15:23.656 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.MIS7cCY7DP 2024-09-11T08:15:23.657 DEBUG:teuthology.orchestra.run.smithi172:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MIS7cCY7DP 2024-09-11T08:15:23.723 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.MIS7cCY7DP 2024-09-11T08:15:23.865 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.MIS7cCY7DP is 6KB 2024-09-11T08:15:23.913 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.MIS7cCY7DP 2024-09-11T08:15:23.928 DEBUG:teuthology.orchestra.run.smithi172:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T08:15:23.990 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T08:15:23.990 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T08:15:23.990 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T08:15:23.990 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T08:15:23.992 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T08:15:24.136 DEBUG:teuthology.orchestra.run.smithi172:> sudo shutdown -r now 2024-09-11T08:15:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:15:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:15:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:15:38.555 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:15:38.555 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:15:38.555 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:15:38.555 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T08:15:38.555 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T08:15:38.556 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T08:15:38.556 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T08:15:38.556 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:15:38.556 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:15:38.904 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:15:38.999 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:15:38.999 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:15:38.999 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T08:15:38.999 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T08:15:38.999 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T08:15:39.000 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T08:15:39.000 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T08:15:39.000 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:15:39.059 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:15:39.059 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:15:39.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-09-11T08:15:39.090 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 82.el9 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8274670 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T08:15:39.091 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:15:39.092 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T08:15:39.093 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T08:15:39.093 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-505.el9.x86_64 2024-09-11T08:15:39.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T08:15:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-11T08:15:40.970 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T08:15:40.985 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-11T08:15:40.987 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-11T08:15:41.002 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.kQIPsxyOZF 2024-09-11T08:15:41.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kQIPsxyOZF 2024-09-11T08:15:41.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.kQIPsxyOZF 2024-09-11T08:15:41.221 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.kQIPsxyOZF is 6KB 2024-09-11T08:15:41.268 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.kQIPsxyOZF 2024-09-11T08:15:41.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T08:15:41.345 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T08:15:41.345 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T08:15:41.345 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T08:15:41.346 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T08:15:41.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T08:15:41.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-11T08:15:54.148 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T08:15:54.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T08:15:54.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:16:11.543 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T08:16:11.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T08:16:11.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:16:12.609 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-09-11T08:16:21.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T08:16:21.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:16:24.669 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-09-11T08:16:30.020 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-09-11T08:16:36.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T08:16:36.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:16:39.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T08:16:39.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:16:55.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-09-11T08:17:10.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T08:17:10.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:17:10.943 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-09-11T08:17:11.399 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T08:17:11.400 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T08:17:11.400 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-09-11T08:17:11.414 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-505.el9.x86_64 2024-09-11T08:17:11.414 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T08:17:11.415 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T08:17:11.415 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T08:17:12.415 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T08:17:12.415 DEBUG:teuthology.orchestra.run.smithi172:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T08:17:12.515 INFO:teuthology.orchestra.run.smithi172.stdout:ttyS1 2024-09-11T08:17:12.534 DEBUG:teuthology.parallel:result is None 2024-09-11T08:17:39.027 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T08:17:51.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T08:17:51.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:17:51.437 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-11T08:17:51.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T08:17:51.781 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T08:17:51.781 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-11T08:17:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-505.el9.x86_64 2024-09-11T08:17:51.798 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T08:17:51.798 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T08:17:51.798 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T08:17:52.799 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T08:17:52.799 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T08:17:52.874 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-09-11T08:17:52.895 DEBUG:teuthology.parallel:result is None 2024-09-11T08:17:52.896 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T08:17:52.904 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T08:17:52.904 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T08:17:52.906 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T08:17:52.928 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T08:17:52.982 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T08:17:53.013 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T08:17:53.014 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T08:17:53.016 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T08:17:53.056 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T08:17:53.062 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T08:17:53.063 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T08:17:53.094 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T08:17:53.125 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:17:53.136 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:17:53.137 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:17:53.148 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:17:53.150 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T08:17:53.157 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T08:17:53.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T08:17:53.159 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T08:17:53.207 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T08:17:53.216 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T08:17:53.216 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T08:17:53.229 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T08:17:53.263 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:17:53.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:17:53.375 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:17:53.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:17:53.503 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:17:53.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T08:17:53.573 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:17:53.622 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:17:53.666 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:17:53.747 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:17:53.791 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:17:53.791 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T08:17:53.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-11T08:17:53.859 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-09-11T08:17:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:17:53.938 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:17:54.264 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T08:17:54.271 INFO:teuthology.task.internal:Starting timer... 2024-09-11T08:17:54.272 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T08:17:54.330 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T08:17:54.368 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T08:17:54.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-09-11T08:17:54.415 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-09-11T08:17:54.417 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-09-11T08:17:54.478 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-09-11T08:17:54.480 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T08:17:54.480 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-09-11T08:17:54.509 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-09-11T08:17:54.510 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-09-11T08:17:54.528 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-09-11T08:17:54.528 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2024-09-11T08:17:54.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:17:54.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:17:54.580 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:17:54.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:17:54.607 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T08:17:54.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-09-11T08:17:54.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-09-11T08:17:54.690 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T08:17:54.701 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T08:17:54.713 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T08:17:54.714 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T08:21:40.450 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2024-09-11T08:21:40.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T08:21:40.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:21:40.569 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-11T08:21:40.649 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T08:21:40.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T08:21:40.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:21:40.723 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-09-11T08:21:40.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T08:21:40.800 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T08:21:40.811 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T08:21:40.811 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:21:40.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:21:40.814 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:21:40.814 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:21:40.843 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:21:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:21:40.868 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:21:40.882 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:21:40.885 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-09-11T08:21:40.896 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-09-11T08:21:40.906 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-09-11T08:21:40.912 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:21:40.918 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-09-11T08:21:40.926 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:21:40.932 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:21:40.947 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:21:40.985 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:21:40.989 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:21:40.989 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-11T08:21:40.989 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:21:40.989 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:21:40.989 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:21:40.990 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:21:41.004 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:21:41.007 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:21:41.007 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-11T08:21:41.007 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:21:41.007 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:21:41.007 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:21:41.007 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:21:41.008 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T08:21:41.017 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T08:21:41.018 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T08:21:41.018 DEBUG:teuthology.orchestra.run.smithi043:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T08:21:41.018 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi172.front.sepia.ceph.com 2024-09-11T08:21:41.018 DEBUG:teuthology.orchestra.run.smithi172:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T08:21:41.400 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:21:41.401 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout:Removing: 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout:Removing unused dependencies: 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout:Remove 2 Packages 2024-09-11T08:21:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:21:41.407 INFO:teuthology.orchestra.run.smithi043.stdout:Freed space: 5.5 M 2024-09-11T08:21:41.407 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:21:41.411 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:21:41.411 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:21:41.417 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:21:41.417 INFO:teuthology.orchestra.run.smithi172.stdout: Package Architecture Version Repository Size 2024-09-11T08:21:41.417 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:21:41.417 INFO:teuthology.orchestra.run.smithi172.stdout:Removing: 2024-09-11T08:21:41.417 INFO:teuthology.orchestra.run.smithi172.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T08:21:41.417 INFO:teuthology.orchestra.run.smithi172.stdout:Removing unused dependencies: 2024-09-11T08:21:41.417 INFO:teuthology.orchestra.run.smithi172.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T08:21:41.418 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:21:41.418 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:21:41.418 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:21:41.418 INFO:teuthology.orchestra.run.smithi172.stdout:Remove 2 Packages 2024-09-11T08:21:41.418 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:21:41.418 INFO:teuthology.orchestra.run.smithi172.stdout:Freed space: 5.5 M 2024-09-11T08:21:41.418 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:21:41.422 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:21:41.422 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:21:41.469 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:21:41.470 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:21:41.484 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:21:41.484 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:21:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:21:41.625 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:21:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T08:21:41.716 INFO:teuthology.orchestra.run.smithi172.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T08:21:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:21:41.776 INFO:teuthology.orchestra.run.smithi172.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:21:42.184 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:21:42.184 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T08:21:42.307 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:21:42.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T08:21:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T08:21:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:21:42.436 INFO:teuthology.orchestra.run.smithi043.stdout:Removed: 2024-09-11T08:21:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:21:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:21:42.436 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:21:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T08:21:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:21:42.545 INFO:teuthology.orchestra.run.smithi172.stdout:Removed: 2024-09-11T08:21:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:21:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:21:42.545 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:21:43.293 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:46 ago on Wed 11 Sep 2024 08:20:57 AM UTC. 2024-09-11T08:21:43.322 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 08:20:51 AM UTC. 2024-09-11T08:21:43.430 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:21:43.431 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:21:43.431 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:21:43.431 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:21:43.431 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:21:43.431 INFO:teuthology.orchestra.run.smithi043.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T08:21:43.431 INFO:teuthology.orchestra.run.smithi043.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T08:21:43.431 INFO:teuthology.orchestra.run.smithi043.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T08:21:43.431 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout:Install 7 Packages 2024-09-11T08:21:43.432 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:21:43.447 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 5.2 M 2024-09-11T08:21:43.447 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 19 M 2024-09-11T08:21:43.447 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:21:43.475 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:21:43.476 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:21:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:21:43.476 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:21:43.477 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:21:43.478 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:21:43.478 INFO:teuthology.orchestra.run.smithi172.stdout:Install 7 Packages 2024-09-11T08:21:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:21:43.509 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 5.2 M 2024-09-11T08:21:43.509 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 19 M 2024-09-11T08:21:43.509 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:21:43.707 INFO:teuthology.orchestra.run.smithi172.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 346 kB/s | 44 kB 00:00 2024-09-11T08:21:43.732 INFO:teuthology.orchestra.run.smithi172.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 692 kB/s | 106 kB 00:00 2024-09-11T08:21:43.783 INFO:teuthology.orchestra.run.smithi172.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-11T08:21:43.808 INFO:teuthology.orchestra.run.smithi172.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 715 kB/s | 72 kB 00:00 2024-09-11T08:21:43.900 INFO:teuthology.orchestra.run.smithi172.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.8 MB/s | 906 kB 00:00 2024-09-11T08:21:43.925 INFO:teuthology.orchestra.run.smithi172.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.8 MB/s | 837 kB 00:00 2024-09-11T08:21:44.158 INFO:teuthology.orchestra.run.smithi043.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 300 kB/s | 44 kB 00:00 2024-09-11T08:21:44.183 INFO:teuthology.orchestra.run.smithi043.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 611 kB/s | 106 kB 00:00 2024-09-11T08:21:44.233 INFO:teuthology.orchestra.run.smithi043.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-11T08:21:44.260 INFO:teuthology.orchestra.run.smithi043.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 714 kB/s | 72 kB 00:00 2024-09-11T08:21:44.292 INFO:teuthology.orchestra.run.smithi043.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.1 MB/s | 906 kB 00:00 2024-09-11T08:21:44.384 INFO:teuthology.orchestra.run.smithi043.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.4 MB/s | 837 kB 00:00 2024-09-11T08:21:44.417 INFO:teuthology.orchestra.run.smithi172.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 5.2 MB/s | 3.2 MB 00:00 2024-09-11T08:21:44.417 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:21:44.417 INFO:teuthology.orchestra.run.smithi172.stdout:Total 5.7 MB/s | 5.2 MB 00:00 2024-09-11T08:21:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:21:44.541 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:21:44.541 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:21:44.843 INFO:teuthology.orchestra.run.smithi043.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 5.4 MB/s | 3.2 MB 00:00 2024-09-11T08:21:44.843 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:21:44.843 INFO:teuthology.orchestra.run.smithi043.stdout:Total 3.7 MB/s | 5.2 MB 00:01 2024-09-11T08:21:44.877 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:21:44.877 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:21:44.962 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:21:44.978 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:21:44.978 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:21:45.281 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:21:45.281 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:21:45.370 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:21:45.462 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T08:21:45.496 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T08:21:45.533 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T08:21:45.744 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T08:21:45.780 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:21:45.784 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:21:45.872 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T08:21:45.911 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T08:21:45.942 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T08:21:46.165 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T08:21:46.198 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:21:46.213 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:21:46.253 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:21:46.362 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:21:46.429 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:21:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:21:46.694 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:21:46.821 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:21:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:21:47.369 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:21:47.369 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T08:21:47.369 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T08:21:47.369 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T08:21:47.369 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T08:21:47.369 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T08:21:47.369 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:21:47.689 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:21:47.864 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:21:47.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T08:21:47.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T08:21:47.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T08:21:47.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T08:21:47.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T08:21:47.865 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T08:21:48.068 DEBUG:teuthology.parallel:result is None 2024-09-11T08:21:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:21:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:21:48.204 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:21:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:21:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T08:21:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T08:21:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T08:21:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:21:48.205 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:21:48.597 DEBUG:teuthology.parallel:result is None 2024-09-11T08:21:48.597 INFO:teuthology.run_tasks:Running task install... 2024-09-11T08:21:48.606 DEBUG:teuthology.task.install:project ceph 2024-09-11T08:21:48.606 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}, 'extra_packages': ['cephadm']} 2024-09-11T08:21:48.606 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T08:21:48.607 INFO:teuthology.task.install:Using flavor: default 2024-09-11T08:21:48.612 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T08:21:48.612 INFO:teuthology.task.install:extra packages: [] 2024-09-11T08:21:48.612 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T08:21:48.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:21:48.615 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T08:21:48.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:21:48.763 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T08:21:48.764 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T08:21:48.767 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T08:21:48.767 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T08:21:48.875 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T08:21:48.876 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:21:48.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T08:21:48.887 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T08:21:48.888 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:21:48.888 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T08:21:48.904 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T08:21:48.904 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T08:21:48.918 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T08:21:48.918 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T08:21:48.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T08:21:48.988 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T08:21:49.050 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T08:21:49.066 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T08:21:49.115 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-09-11T08:21:49.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-09-11T08:21:49.132 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-09-11T08:21:49.134 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-09-11T08:21:49.356 INFO:teuthology.orchestra.run.smithi043.stdout:45 files removed 2024-09-11T08:21:49.360 INFO:teuthology.orchestra.run.smithi172.stdout:45 files removed 2024-09-11T08:21:49.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2024-09-11T08:21:49.388 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw 2024-09-11T08:21:49.948 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 283 kB/s | 77 kB 00:00 2024-09-11T08:21:49.966 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 274 kB/s | 77 kB 00:00 2024-09-11T08:21:50.240 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 44 kB/s | 11 kB 00:00 2024-09-11T08:21:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 44 kB/s | 11 kB 00:00 2024-09-11T08:21:50.516 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 7.4 kB/s | 1.8 kB 00:00 2024-09-11T08:21:50.525 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 7.8 kB/s | 1.8 kB 00:00 2024-09-11T08:21:52.109 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.2 MB 00:01 2024-09-11T08:21:52.208 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-11T08:21:53.992 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-11T08:21:58.159 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-11T08:21:59.785 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 19 kB 00:00 2024-09-11T08:22:00.548 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-11T08:22:00.751 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-11T08:22:05.269 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-11T08:22:05.937 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-11T08:22:06.736 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 95 kB/s | 19 kB 00:00 2024-09-11T08:22:07.881 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T08:22:08.317 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:22:08.318 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:08.318 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:22:08.318 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T08:22:08.319 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T08:22:08.320 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout:Install 22 Packages 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 2 Packages 2024-09-11T08:22:08.321 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:08.322 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 58 M 2024-09-11T08:22:08.322 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:22:09.196 INFO:teuthology.orchestra.run.smithi172.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-11T08:22:09.247 INFO:teuthology.orchestra.run.smithi172.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 503 kB/s | 25 kB 00:00 2024-09-11T08:22:09.339 INFO:teuthology.orchestra.run.smithi172.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 6.8 MB/s | 636 kB 00:00 2024-09-11T08:22:09.439 INFO:teuthology.orchestra.run.smithi172.stdout:(4/24): libradosstriper1-17.2.7-1623.gd0e6828a. 4.5 MB/s | 457 kB 00:00 2024-09-11T08:22:09.781 INFO:teuthology.orchestra.run.smithi172.stdout:(5/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-11T08:22:10.023 INFO:teuthology.orchestra.run.smithi172.stdout:(6/24): ceph-common-17.2.7-1623.gd0e6828a.el9.x 18 MB/s | 23 MB 00:01 2024-09-11T08:22:10.048 INFO:teuthology.orchestra.run.smithi172.stdout:(7/24): python3-ceph-argparse-17.2.7-1623.gd0e6 169 kB/s | 45 kB 00:00 2024-09-11T08:22:10.073 INFO:teuthology.orchestra.run.smithi172.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 2.4 MB/s | 120 kB 00:00 2024-09-11T08:22:10.107 INFO:teuthology.orchestra.run.smithi172.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 2.6 MB/s | 156 kB 00:00 2024-09-11T08:22:10.132 INFO:teuthology.orchestra.run.smithi172.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 5.3 MB/s | 320 kB 00:00 2024-09-11T08:22:10.157 INFO:teuthology.orchestra.run.smithi172.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 5.7 MB/s | 294 kB 00:00 2024-09-11T08:22:10.182 INFO:teuthology.orchestra.run.smithi172.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.9 MB/s | 99 kB 00:00 2024-09-11T08:22:10.341 INFO:teuthology.orchestra.run.smithi172.stdout:(13/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9 9.2 MB/s | 14 MB 00:01 2024-09-11T08:22:10.416 INFO:teuthology.orchestra.run.smithi172.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 194 kB/s | 45 kB 00:00 2024-09-11T08:22:10.500 INFO:teuthology.orchestra.run.smithi172.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 468 kB/s | 160 kB 00:00 2024-09-11T08:22:10.550 INFO:teuthology.orchestra.run.smithi172.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-11T08:22:10.567 INFO:teuthology.orchestra.run.smithi172.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 633 kB/s | 42 kB 00:00 2024-09-11T08:22:10.617 INFO:teuthology.orchestra.run.smithi172.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-09-11T08:22:10.651 INFO:teuthology.orchestra.run.smithi172.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 580 kB/s | 49 kB 00:00 2024-09-11T08:22:10.676 INFO:teuthology.orchestra.run.smithi172.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2024-09-11T08:22:10.702 INFO:teuthology.orchestra.run.smithi172.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.0 MB/s | 309 kB 00:00 2024-09-11T08:22:10.752 INFO:teuthology.orchestra.run.smithi172.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T08:22:10.877 INFO:teuthology.orchestra.run.smithi172.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 15 MB/s | 3.1 MB 00:00 2024-09-11T08:22:10.935 INFO:teuthology.orchestra.run.smithi172.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 13 MB/s | 2.9 MB 00:00 2024-09-11T08:22:10.940 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:10.941 INFO:teuthology.orchestra.run.smithi172.stdout:Total 22 MB/s | 58 MB 00:02 2024-09-11T08:22:11.001 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:22:11.035 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:22:11.035 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:22:11.511 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:22:11.511 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:22:12.710 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:22:13.081 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T08:22:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:22:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:22:13.325 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:22:13.356 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:22:13.444 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:22:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T08:22:13.519 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T08:22:13.654 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T08:22:13.964 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T08:22:13.992 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:22:14.055 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:22:14.092 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T08:22:14.134 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T08:22:14.171 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:22:14.234 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:22:14.270 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:22:14.336 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:22:14.376 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T08:22:14.420 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T08:22:14.453 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T08:22:14.747 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T08:22:14.791 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:22:14.865 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:22:14.946 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T08:22:14.976 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T08:22:16.127 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T08:22:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T08:22:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout:Install 22 Packages 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-09-11T08:22:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:16.132 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 58 M 2024-09-11T08:22:16.132 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:22:17.072 INFO:teuthology.orchestra.run.smithi043.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 10 MB/s | 5.0 MB 00:00 2024-09-11T08:22:17.139 INFO:teuthology.orchestra.run.smithi043.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 378 kB/s | 25 kB 00:00 2024-09-11T08:22:17.247 INFO:teuthology.orchestra.run.smithi043.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 5.8 MB/s | 636 kB 00:00 2024-09-11T08:22:17.347 INFO:teuthology.orchestra.run.smithi043.stdout:(4/24): libradosstriper1-17.2.7-1623.gd0e6828a. 4.5 MB/s | 457 kB 00:00 2024-09-11T08:22:17.461 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:22:17.498 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:22:17.573 INFO:teuthology.orchestra.run.smithi043.stdout:(5/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 15 MB/s | 14 MB 00:00 2024-09-11T08:22:17.623 INFO:teuthology.orchestra.run.smithi043.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 903 kB/s | 45 kB 00:00 2024-09-11T08:22:17.707 INFO:teuthology.orchestra.run.smithi043.stdout:(7/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-11T08:22:17.741 INFO:teuthology.orchestra.run.smithi043.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 1.0 MB/s | 120 kB 00:00 2024-09-11T08:22:17.766 INFO:teuthology.orchestra.run.smithi043.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 2.6 MB/s | 156 kB 00:00 2024-09-11T08:22:17.791 INFO:teuthology.orchestra.run.smithi043.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 6.2 MB/s | 320 kB 00:00 2024-09-11T08:22:18.008 INFO:teuthology.orchestra.run.smithi043.stdout:(11/24): ceph-common-17.2.7-1623.gd0e6828a.el9. 16 MB/s | 23 MB 00:01 2024-09-11T08:22:18.014 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:22:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 411 kB/s | 99 kB 00:00 2024-09-11T08:22:18.051 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:22:18.067 INFO:teuthology.orchestra.run.smithi043.stdout:(13/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 978 kB/s | 294 kB 00:00 2024-09-11T08:22:18.142 INFO:teuthology.orchestra.run.smithi043.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-11T08:22:18.150 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:22:18.150 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T08:22:18.150 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T08:22:18.150 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:18.167 INFO:teuthology.orchestra.run.smithi043.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 338 kB/s | 45 kB 00:00 2024-09-11T08:22:18.185 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:22:18.218 INFO:teuthology.orchestra.run.smithi043.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-11T08:22:18.243 INFO:teuthology.orchestra.run.smithi043.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 561 kB/s | 42 kB 00:00 2024-09-11T08:22:18.293 INFO:teuthology.orchestra.run.smithi043.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-11T08:22:18.327 INFO:teuthology.orchestra.run.smithi043.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-09-11T08:22:18.360 INFO:teuthology.orchestra.run.smithi043.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.1 MB/s | 309 kB 00:00 2024-09-11T08:22:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 686 kB/s | 67 kB 00:00 2024-09-11T08:22:18.444 INFO:teuthology.orchestra.run.smithi043.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T08:22:18.586 INFO:teuthology.orchestra.run.smithi043.stdout:(23/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-09-11T08:22:18.644 INFO:teuthology.orchestra.run.smithi043.stdout:(24/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-11T08:22:18.650 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:18.650 INFO:teuthology.orchestra.run.smithi043.stdout:Total 23 MB/s | 58 MB 00:02 2024-09-11T08:22:18.718 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:22:18.755 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:22:18.756 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:22:19.255 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:22:19.255 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:22:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:22:20.888 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T08:22:20.930 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:22:21.023 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:22:21.111 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:22:21.165 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:22:21.254 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:22:21.322 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T08:22:21.362 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T08:22:21.827 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T08:22:21.859 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:22:21.923 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:22:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T08:22:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T08:22:22.049 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:22:22.112 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:22:22.145 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:22:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:22:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T08:22:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T08:22:22.337 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T08:22:22.646 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T08:22:22.691 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:22:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:22:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T08:22:22.885 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T08:22:25.558 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:22:25.598 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:22:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:22:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:22:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:22:26.360 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T08:22:26.361 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T08:22:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-09-11T08:22:28.204 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:29.543 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:22:29.605 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:22:29.606 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:22:29.606 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T08:22:29.606 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:22:29.606 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:22:29.606 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:29.607 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:22:29.704 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:22:29.705 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:22:30.925 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:22:30.925 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T08:22:30.925 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:22:30.925 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:22:30.925 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:22:30.925 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T08:22:30.926 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T08:22:30.926 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T08:22:30.926 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T08:22:30.926 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T08:22:30.926 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T08:22:30.926 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:22:30.926 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:22:30.926 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T08:22:30.927 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T08:22:31.741 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:22:31.741 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:31.741 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-09-11T08:22:31.741 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.741 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T08:22:31.742 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:31.743 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:22:32.079 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-test 2024-09-11T08:22:32.652 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:27 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:22:32.791 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:22:32.792 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:32.792 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:22:32.792 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:32.792 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:22:32.792 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout:Install 4 Packages 2024-09-11T08:22:32.793 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:32.794 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 48 M 2024-09-11T08:22:32.794 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 193 M 2024-09-11T08:22:32.794 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:22:33.041 INFO:teuthology.orchestra.run.smithi172.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-11T08:22:33.066 INFO:teuthology.orchestra.run.smithi172.stdout:(2/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 721 kB/s | 163 kB 00:00 2024-09-11T08:22:33.091 INFO:teuthology.orchestra.run.smithi172.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T08:22:34.308 INFO:teuthology.orchestra.run.smithi172.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 33 MB/s | 48 MB 00:01 2024-09-11T08:22:34.310 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:34.310 INFO:teuthology.orchestra.run.smithi172.stdout:Total 32 MB/s | 48 MB 00:01 2024-09-11T08:22:34.338 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:22:34.357 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:22:34.357 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:22:34.726 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:22:34.727 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:22:35.273 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:22:35.352 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T08:22:35.390 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T08:22:35.418 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-09-11T08:22:36.970 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:38.336 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:22:38.416 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:22:38.416 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:22:38.416 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T08:22:38.416 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:22:38.416 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:22:38.416 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:38.418 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:22:38.506 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:22:38.506 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:22:39.764 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:22:39.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T08:22:39.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T08:22:39.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T08:22:39.865 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:22:39.949 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:22:40.634 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.635 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:40.637 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T08:22:40.637 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:40.637 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:22:41.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2024-09-11T08:22:41.692 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:22:41.835 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:22:41.835 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:41.836 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-11T08:22:41.837 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:41.837 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 48 M 2024-09-11T08:22:41.837 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 193 M 2024-09-11T08:22:41.837 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:22:42.084 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-11T08:22:42.126 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-11T08:22:42.184 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 660 kB/s | 163 kB 00:00 2024-09-11T08:22:42.196 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:22:42.196 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T08:22:42.196 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T08:22:42.196 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T08:22:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T08:22:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:42.536 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:22:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T08:22:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T08:22:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:42.537 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:22:42.819 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph 2024-09-11T08:22:43.385 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:38 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:22:43.443 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T08:22:43.445 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:43.445 INFO:teuthology.orchestra.run.smithi043.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-11T08:22:43.474 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:22:43.495 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:22:43.495 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:22:43.538 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:22:43.540 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:22:43.540 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:43.540 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:22:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T08:22:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T08:22:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:22:43.543 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:43.544 INFO:teuthology.orchestra.run.smithi172.stdout:Install 34 Packages 2024-09-11T08:22:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:43.544 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 27 M 2024-09-11T08:22:43.544 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 96 M 2024-09-11T08:22:43.544 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:22:43.882 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:22:43.882 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:22:44.103 INFO:teuthology.orchestra.run.smithi172.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 48 kB/s | 6.5 kB 00:00 2024-09-11T08:22:44.303 INFO:teuthology.orchestra.run.smithi172.stdout:(2/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.2 MB/s | 1.4 MB 00:00 2024-09-11T08:22:44.354 INFO:teuthology.orchestra.run.smithi172.stdout:(3/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 5.4 MB/s | 2.1 MB 00:00 2024-09-11T08:22:44.437 INFO:teuthology.orchestra.run.smithi172.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 2.8 MB/s | 240 kB 00:00 2024-09-11T08:22:44.512 INFO:teuthology.orchestra.run.smithi172.stdout:(5/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 9.6 MB/s | 3.9 MB 00:00 2024-09-11T08:22:44.556 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:22:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T08:22:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T08:22:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:22:44.813 INFO:teuthology.orchestra.run.smithi172.stdout:(6/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 30 MB/s | 15 MB 00:00 2024-09-11T08:22:44.838 INFO:teuthology.orchestra.run.smithi172.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 633 kB/s | 253 kB 00:00 2024-09-11T08:22:44.913 INFO:teuthology.orchestra.run.smithi172.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-11T08:22:44.955 INFO:teuthology.orchestra.run.smithi172.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 2.8 MB/s | 1.2 MB 00:00 2024-09-11T08:22:44.980 INFO:teuthology.orchestra.run.smithi172.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 1.8 MB/s | 125 kB 00:00 2024-09-11T08:22:45.014 INFO:teuthology.orchestra.run.smithi172.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-11T08:22:45.039 INFO:teuthology.orchestra.run.smithi172.stdout:(12/34): python3-ply-3.11-14.el9.noarch.rpm 471 kB/s | 106 kB 00:00 2024-09-11T08:22:45.123 INFO:teuthology.orchestra.run.smithi172.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 320 kB/s | 35 kB 00:00 2024-09-11T08:22:45.148 INFO:teuthology.orchestra.run.smithi172.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-11T08:22:45.182 INFO:teuthology.orchestra.run.smithi172.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 543 kB/s | 77 kB 00:00 2024-09-11T08:22:45.207 INFO:teuthology.orchestra.run.smithi172.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 709 kB/s | 42 kB 00:00 2024-09-11T08:22:45.232 INFO:teuthology.orchestra.run.smithi172.stdout:(17/34): python3-pytz-2021.1-5.el9.noarch.rpm 467 kB/s | 51 kB 00:00 2024-09-11T08:22:45.282 INFO:teuthology.orchestra.run.smithi172.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-11T08:22:45.308 INFO:teuthology.orchestra.run.smithi172.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 461 kB/s | 11 kB 00:00 2024-09-11T08:22:45.333 INFO:teuthology.orchestra.run.smithi172.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T08:22:45.358 INFO:teuthology.orchestra.run.smithi172.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-11T08:22:45.383 INFO:teuthology.orchestra.run.smithi172.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-11T08:22:45.409 INFO:teuthology.orchestra.run.smithi172.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-11T08:22:45.434 INFO:teuthology.orchestra.run.smithi172.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-11T08:22:45.459 INFO:teuthology.orchestra.run.smithi172.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-11T08:22:45.484 INFO:teuthology.orchestra.run.smithi172.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-11T08:22:45.510 INFO:teuthology.orchestra.run.smithi172.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-11T08:22:45.526 INFO:teuthology.orchestra.run.smithi172.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2024-09-11T08:22:45.560 INFO:teuthology.orchestra.run.smithi172.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-11T08:22:45.577 INFO:teuthology.orchestra.run.smithi172.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-11T08:22:45.602 INFO:teuthology.orchestra.run.smithi172.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-11T08:22:45.619 INFO:teuthology.orchestra.run.smithi172.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-11T08:22:45.653 INFO:teuthology.orchestra.run.smithi172.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T08:22:45.678 INFO:teuthology.orchestra.run.smithi172.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-11T08:22:45.681 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:45.682 INFO:teuthology.orchestra.run.smithi172.stdout:Total 13 MB/s | 27 MB 00:02 2024-09-11T08:22:45.805 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:22:45.844 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:22:45.844 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:22:46.250 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:22:46.250 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:22:47.479 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:22:47.539 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T08:22:47.572 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T08:22:47.622 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T08:22:47.713 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T08:22:47.799 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T08:22:47.840 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T08:22:47.875 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T08:22:47.905 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T08:22:47.940 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T08:22:47.973 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T08:22:48.014 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T08:22:48.056 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T08:22:48.091 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T08:22:48.157 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T08:22:48.197 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:22:48.269 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T08:22:48.407 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T08:22:48.488 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T08:22:48.541 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T08:22:48.580 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T08:22:48.637 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T08:22:48.700 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T08:22:48.887 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T08:22:48.936 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T08:22:49.014 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T08:22:49.101 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T08:22:49.139 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T08:22:49.266 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T08:22:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:22:49.423 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T08:22:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:22:49.453 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:22:49.479 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:22:49.479 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:22:49.479 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T08:22:49.479 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:22:49.479 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:22:49.479 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:50.839 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:22:50.868 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:22:50.868 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:22:50.868 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T08:22:50.868 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:22:50.869 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:22:50.869 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:51.248 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:22:51.272 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:22:51.272 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:22:51.272 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T08:22:51.272 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:22:51.272 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:22:51.272 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:51.473 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:22:51.497 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:22:51.497 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:22:51.497 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T08:22:51.498 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:22:51.498 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:22:51.498 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:51.561 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:22:51.659 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:22:51.660 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T08:22:51.660 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T08:22:51.660 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T08:22:51.960 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T08:22:51.960 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:51.960 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:22:51.960 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:51.961 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:51.961 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T08:22:51.961 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T08:22:51.961 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:51.961 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:22:52.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2024-09-11T08:22:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:39 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:22:52.994 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:22:52.996 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T08:22:52.997 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T08:22:52.998 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:52.999 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:22:53.000 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:22:53.000 INFO:teuthology.orchestra.run.smithi043.stdout:Install 34 Packages 2024-09-11T08:22:53.000 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:53.000 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 27 M 2024-09-11T08:22:53.000 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 96 M 2024-09-11T08:22:53.000 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:22:53.260 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T08:22:53.261 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T08:22:53.262 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T08:22:53.263 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T08:22:53.263 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T08:22:53.263 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T08:22:53.263 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T08:22:53.263 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T08:22:53.263 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T08:22:53.263 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T08:22:53.263 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T08:22:53.355 INFO:teuthology.orchestra.run.smithi043.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-11T08:22:53.522 INFO:teuthology.orchestra.run.smithi043.stdout:(2/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 7.0 MB/s | 2.1 MB 00:00 2024-09-11T08:22:53.581 INFO:teuthology.orchestra.run.smithi043.stdout:(3/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-09-11T08:22:53.657 INFO:teuthology.orchestra.run.smithi043.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 3.1 MB/s | 240 kB 00:00 2024-09-11T08:22:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:(5/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 9.8 MB/s | 3.9 MB 00:00 2024-09-11T08:22:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-11T08:22:54.032 INFO:teuthology.orchestra.run.smithi043.stdout:(7/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 30 MB/s | 15 MB 00:00 2024-09-11T08:22:54.057 INFO:teuthology.orchestra.run.smithi043.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-09-11T08:22:54.099 INFO:teuthology.orchestra.run.smithi043.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 3.6 MB/s | 1.2 MB 00:00 2024-09-11T08:22:54.141 INFO:teuthology.orchestra.run.smithi043.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 125 kB 00:00 2024-09-11T08:22:54.175 INFO:teuthology.orchestra.run.smithi043.stdout:(11/34): python3-pycparser-2.20-6.el9.noarch.rp 949 kB/s | 135 kB 00:00 2024-09-11T08:22:54.200 INFO:teuthology.orchestra.run.smithi043.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-09-11T08:22:54.284 INFO:teuthology.orchestra.run.smithi043.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 320 kB/s | 35 kB 00:00 2024-09-11T08:22:54.316 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T08:22:54.316 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:54.316 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:22:54.316 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:54.316 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:54.316 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:54.316 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T08:22:54.317 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T08:22:54.318 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:22:54.319 INFO:teuthology.orchestra.run.smithi043.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 979 kB/s | 172 kB 00:00 2024-09-11T08:22:54.343 INFO:teuthology.orchestra.run.smithi043.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 542 kB/s | 77 kB 00:00 2024-09-11T08:22:54.369 INFO:teuthology.orchestra.run.smithi043.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 606 kB/s | 51 kB 00:00 2024-09-11T08:22:54.385 INFO:teuthology.orchestra.run.smithi043.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 619 kB/s | 42 kB 00:00 2024-09-11T08:22:54.427 INFO:teuthology.orchestra.run.smithi043.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 518 kB/s | 43 kB 00:00 2024-09-11T08:22:54.444 INFO:teuthology.orchestra.run.smithi043.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 725 kB/s | 11 kB 00:00 2024-09-11T08:22:54.461 INFO:teuthology.orchestra.run.smithi043.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 1.0 MB/s | 18 kB 00:00 2024-09-11T08:22:54.494 INFO:teuthology.orchestra.run.smithi043.stdout:(21/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T08:22:54.511 INFO:teuthology.orchestra.run.smithi043.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-11T08:22:54.528 INFO:teuthology.orchestra.run.smithi043.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 577 kB/s | 19 kB 00:00 2024-09-11T08:22:54.554 INFO:teuthology.orchestra.run.smithi043.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 471 kB/s | 20 kB 00:00 2024-09-11T08:22:54.587 INFO:teuthology.orchestra.run.smithi043.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 792 kB/s | 46 kB 00:00 2024-09-11T08:22:54.612 INFO:teuthology.orchestra.run.smithi043.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-11T08:22:54.638 INFO:teuthology.orchestra.run.smithi043.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-11T08:22:54.663 INFO:teuthology.orchestra.run.smithi043.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 325 kB/s | 16 kB 00:00 2024-09-11T08:22:54.689 INFO:teuthology.orchestra.run.smithi043.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-11T08:22:54.705 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-base 2024-09-11T08:22:54.714 INFO:teuthology.orchestra.run.smithi043.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-11T08:22:54.739 INFO:teuthology.orchestra.run.smithi043.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-11T08:22:54.764 INFO:teuthology.orchestra.run.smithi043.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 797 kB/s | 20 kB 00:00 2024-09-11T08:22:54.789 INFO:teuthology.orchestra.run.smithi043.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T08:22:54.815 INFO:teuthology.orchestra.run.smithi043.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-11T08:22:54.819 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:54.819 INFO:teuthology.orchestra.run.smithi043.stdout:Total 15 MB/s | 27 MB 00:01 2024-09-11T08:22:54.948 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:22:54.989 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:22:54.989 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:22:55.278 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:22:55.366 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:22:55.366 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:22:55.370 INFO:teuthology.orchestra.run.smithi172.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:22:55.412 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:22:55.413 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:22:55.413 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:22:55.470 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install cephadm 2024-09-11T08:22:56.031 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:22:56.164 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:22:56.164 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout:Install 1 Package 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 83 k 2024-09-11T08:22:56.165 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 375 k 2024-09-11T08:22:56.166 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:22:56.355 INFO:teuthology.orchestra.run.smithi172.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 435 kB/s | 83 kB 00:00 2024-09-11T08:22:56.356 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:56.356 INFO:teuthology.orchestra.run.smithi172.stdout:Total 431 kB/s | 83 kB 00:00 2024-09-11T08:22:56.356 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:22:56.362 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:22:56.362 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:22:56.363 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:22:56.364 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:22:56.442 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:22:56.658 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:22:56.718 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T08:22:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T08:22:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T08:22:56.796 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:22:56.883 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:22:56.887 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T08:22:56.973 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T08:22:57.020 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T08:22:57.055 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T08:22:57.101 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T08:22:57.144 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T08:22:57.177 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T08:22:57.218 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T08:22:57.219 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:22:57.270 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T08:22:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T08:22:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T08:22:57.402 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:22:57.471 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:22:57.471 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:57.471 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:22:57.471 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:22:57.471 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:57.471 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:22:57.478 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T08:22:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T08:22:57.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T08:22:57.713 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T08:22:57.771 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T08:22:57.809 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T08:22:57.868 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T08:22:57.931 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T08:22:58.126 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T08:22:58.197 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T08:22:58.275 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:22:58.276 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T08:22:58.374 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T08:22:58.409 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:22:58.410 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:58.410 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repo Size 2024-09-11T08:22:58.410 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:58.410 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:22:58.410 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T08:22:58.410 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:58.410 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:22:58.411 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:22:58.411 INFO:teuthology.orchestra.run.smithi172.stdout:Install 1 Package 2024-09-11T08:22:58.411 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:58.411 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 137 k 2024-09-11T08:22:58.411 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 379 k 2024-09-11T08:22:58.411 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:22:58.422 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T08:22:58.552 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T08:22:58.611 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 685 kB/s | 137 kB 00:00 2024-09-11T08:22:58.612 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:58.612 INFO:teuthology.orchestra.run.smithi172.stdout:Total 680 kB/s | 137 kB 00:00 2024-09-11T08:22:58.613 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:22:58.620 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:22:58.620 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:22:58.679 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:22:58.679 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:22:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T08:22:58.730 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:22:58.760 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:22:58.761 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:22:58.761 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T08:22:58.761 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:22:58.761 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:22:58.761 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:22:58.849 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:22:58.946 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:22:58.970 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:22:58.970 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:22:58.970 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T08:22:58.971 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:59.936 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:22:59.936 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:59.936 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:22:59.936 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:22:59.936 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:22:59.936 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:00.156 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:23:00.188 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:23:00.189 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:23:00.189 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T08:23:00.189 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:23:00.189 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:23:00.189 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:00.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-mgr 2024-09-11T08:23:00.582 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:23:00.607 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:23:00.608 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:23:00.608 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T08:23:00.608 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:23:00.608 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:23:00.608 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:00.770 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:00.816 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:23:00.841 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:23:00.841 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:23:00.842 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T08:23:00.842 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:23:00.842 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:23:00.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:00.863 INFO:teuthology.orchestra.run.smithi172.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:00.906 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:00.907 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:23:00.907 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:23:00.908 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:00.979 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T08:23:01.547 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:01.681 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:01.682 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:01.682 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:23:01.682 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:01.682 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T08:23:01.683 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:01.684 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:01.684 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:01.684 INFO:teuthology.orchestra.run.smithi172.stdout:Install 7 Packages 2024-09-11T08:23:01.684 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:01.684 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 3.7 M 2024-09-11T08:23:01.684 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 80 M 2024-09-11T08:23:01.685 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:02.199 INFO:teuthology.orchestra.run.smithi172.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 169 kB/s | 24 kB 00:00 2024-09-11T08:23:02.216 INFO:teuthology.orchestra.run.smithi172.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 91 kB/s | 15 kB 00:00 2024-09-11T08:23:02.249 INFO:teuthology.orchestra.run.smithi172.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 180 kB/s | 9.0 kB 00:00 2024-09-11T08:23:02.275 INFO:teuthology.orchestra.run.smithi172.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T08:23:02.292 INFO:teuthology.orchestra.run.smithi172.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 542 kB/s | 41 kB 00:00 2024-09-11T08:23:02.325 INFO:teuthology.orchestra.run.smithi172.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T08:23:02.434 INFO:teuthology.orchestra.run.smithi172.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.9 MB/s | 3.4 MB 00:00 2024-09-11T08:23:02.437 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:02.437 INFO:teuthology.orchestra.run.smithi172.stdout:Total 4.8 MB/s | 3.7 MB 00:00 2024-09-11T08:23:02.438 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:02.446 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:02.446 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:02.524 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:02.525 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T08:23:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T08:23:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T08:23:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T08:23:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T08:23:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T08:23:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T08:23:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T08:23:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T08:23:02.830 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:02.893 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T08:23:02.928 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T08:23:02.983 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T08:23:03.031 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T08:23:03.060 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T08:23:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T08:23:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:03.714 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T08:23:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T08:23:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T08:23:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T08:23:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T08:23:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T08:23:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T08:23:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:03.718 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:04.064 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T08:23:04.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-base 2024-09-11T08:23:04.106 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:23:04.699 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:04.792 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:04.836 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:04.837 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:04.837 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:04.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install cephadm 2024-09-11T08:23:05.112 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:23:05.113 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T08:23:05.113 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T08:23:05.113 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T08:23:05.113 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T08:23:05.113 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T08:23:05.113 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T08:23:05.402 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:05.403 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:05.489 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:05.626 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:05.628 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 83 k 2024-09-11T08:23:05.629 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 375 k 2024-09-11T08:23:05.629 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:05.629 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T08:23:05.817 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 439 kB/s | 83 kB 00:00 2024-09-11T08:23:05.817 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:05.817 INFO:teuthology.orchestra.run.smithi043.stdout:Total 435 kB/s | 83 kB 00:00 2024-09-11T08:23:05.818 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:05.823 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:05.823 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:05.825 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:05.825 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:06.190 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:01 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:06.324 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:06.325 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-09-11T08:23:06.325 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:23:06.325 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-09-11T08:23:06.326 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:06.326 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T08:23:06.326 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:23:06.326 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T08:23:06.326 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T08:23:06.326 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T08:23:06.327 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T08:23:06.327 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T08:23:06.327 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T08:23:06.327 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T08:23:06.327 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T08:23:06.327 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T08:23:06.328 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T08:23:06.328 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T08:23:06.328 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:06.328 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:06.328 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-09-11T08:23:06.328 INFO:teuthology.orchestra.run.smithi172.stdout:Install 12 Packages 2024-09-11T08:23:06.328 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:06.329 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 38 M 2024-09-11T08:23:06.329 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 212 M 2024-09-11T08:23:06.329 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:06.376 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:06.481 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:06.839 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:07.075 INFO:teuthology.orchestra.run.smithi172.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 982 kB/s | 191 kB 00:00 2024-09-11T08:23:07.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:07.094 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:07.094 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:07.094 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:07.094 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:07.094 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:07.167 INFO:teuthology.orchestra.run.smithi172.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-11T08:23:07.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T08:23:07.359 INFO:teuthology.orchestra.run.smithi172.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-11T08:23:07.384 INFO:teuthology.orchestra.run.smithi172.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 106 kB/s | 33 kB 00:00 2024-09-11T08:23:07.592 INFO:teuthology.orchestra.run.smithi172.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 185 kB/s | 38 kB 00:00 2024-09-11T08:23:07.651 INFO:teuthology.orchestra.run.smithi172.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 62 kB/s | 18 kB 00:00 2024-09-11T08:23:07.892 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:08.031 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:08.032 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:08.032 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-11T08:23:08.032 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:08.032 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:08.032 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T08:23:08.032 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:08.032 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:08.033 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:08.033 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-11T08:23:08.033 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:08.033 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 137 k 2024-09-11T08:23:08.033 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 379 k 2024-09-11T08:23:08.033 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:08.118 INFO:teuthology.orchestra.run.smithi172.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 524 kB/s | 245 kB 00:00 2024-09-11T08:23:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 666 kB/s | 137 kB 00:00 2024-09-11T08:23:08.240 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:08.240 INFO:teuthology.orchestra.run.smithi043.stdout:Total 661 kB/s | 137 kB 00:00 2024-09-11T08:23:08.240 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:08.248 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:08.248 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:08.310 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:08.310 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:08.393 INFO:teuthology.orchestra.run.smithi172.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-09-11T08:23:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:08.510 INFO:teuthology.orchestra.run.smithi172.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-11T08:23:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:23:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:23:08.558 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:23:08.559 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T08:23:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:08.602 INFO:teuthology.orchestra.run.smithi172.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.9 MB/s | 4.9 MB 00:01 2024-09-11T08:23:08.894 INFO:teuthology.orchestra.run.smithi172.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.6 MB/s | 5.1 MB 00:00 2024-09-11T08:23:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:23:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:09.508 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:09.508 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:09.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2024-09-11T08:23:10.309 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:57 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:10.403 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:10.447 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:10.448 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:10.448 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:10.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T08:23:11.098 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:58 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:11.238 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:11.238 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-09-11T08:23:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T08:23:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:11.240 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:11.240 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:11.240 INFO:teuthology.orchestra.run.smithi043.stdout:Install 7 Packages 2024-09-11T08:23:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:11.240 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.7 M 2024-09-11T08:23:11.241 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 80 M 2024-09-11T08:23:11.241 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:11.726 INFO:teuthology.orchestra.run.smithi043.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 171 kB/s | 24 kB 00:00 2024-09-11T08:23:11.743 INFO:teuthology.orchestra.run.smithi043.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 92 kB/s | 15 kB 00:00 2024-09-11T08:23:11.785 INFO:teuthology.orchestra.run.smithi043.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 154 kB/s | 9.0 kB 00:00 2024-09-11T08:23:11.810 INFO:teuthology.orchestra.run.smithi043.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T08:23:11.835 INFO:teuthology.orchestra.run.smithi043.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-11T08:23:11.869 INFO:teuthology.orchestra.run.smithi043.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-09-11T08:23:11.991 INFO:teuthology.orchestra.run.smithi043.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.2 MB/s | 3.4 MB 00:00 2024-09-11T08:23:11.995 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:11.995 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.8 MB/s | 3.7 MB 00:00 2024-09-11T08:23:11.995 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:12.004 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:12.005 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:12.108 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:12.108 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:12.426 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:12.496 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T08:23:12.530 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T08:23:12.586 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T08:23:12.616 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T08:23:12.645 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T08:23:13.708 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T08:23:13.749 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:23:14.875 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:23:14.875 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T08:23:14.875 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T08:23:14.875 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T08:23:14.876 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T08:23:14.876 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T08:23:14.876 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T08:23:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T08:23:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T08:23:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:15.160 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:15.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T08:23:15.968 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:02 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:16.108 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:16.109 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-11T08:23:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T08:23:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T08:23:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T08:23:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T08:23:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:16.111 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:16.111 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-11T08:23:16.111 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-11T08:23:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:16.112 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 38 M 2024-09-11T08:23:16.112 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 212 M 2024-09-11T08:23:16.112 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:16.386 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-11T08:23:16.469 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-11T08:23:16.494 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-11T08:23:16.536 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-09-11T08:23:16.612 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 514 kB/s | 38 kB 00:00 2024-09-11T08:23:16.770 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-16 14 MB/s | 7.4 MB 00:00 2024-09-11T08:23:16.829 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.3 MB/s | 3.0 MB 00:00 2024-09-11T08:23:16.963 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 14 MB/s | 4.9 MB 00:00 2024-09-11T08:23:17.029 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-09-11T08:23:17.055 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 861 kB/s | 245 kB 00:00 2024-09-11T08:23:17.080 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-11T08:23:17.330 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 54 MB/s | 16 MB 00:00 2024-09-11T08:23:17.331 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:17.332 INFO:teuthology.orchestra.run.smithi043.stdout:Total 31 MB/s | 38 MB 00:01 2024-09-11T08:23:17.698 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:17.725 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:17.725 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:18.259 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:18.260 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:18.543 INFO:teuthology.orchestra.run.smithi172.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.6 MB/s | 16 MB 00:10 2024-09-11T08:23:18.543 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:18.544 INFO:teuthology.orchestra.run.smithi172.stdout:Total 3.1 MB/s | 38 MB 00:12 2024-09-11T08:23:18.896 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:18.923 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:18.923 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:23:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:23:19.283 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:23:19.322 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:23:19.445 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:19.445 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:19.514 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:23:19.545 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:23:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:23:20.206 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:20.284 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:23:20.308 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:23:20.324 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:23:20.421 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:23:20.425 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:23:20.475 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:23:20.660 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:23:20.699 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:23:20.776 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:23:21.421 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:23:21.546 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:23:21.753 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:23:22.819 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:23:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:23:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:23:23.871 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:23:23.906 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:23:25.627 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:23:25.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T08:23:25.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:23:25.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:23:25.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:23:25.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:23:25.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:23:25.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:23:25.628 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:23:25.628 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:23:25.628 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:23:25.628 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:23:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:23:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:23:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:23:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:23:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:23:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:23:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:26.068 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:26.465 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:23:26.465 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T08:23:26.465 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:23:26.465 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:23:26.466 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:23:26.466 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:23:26.466 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:23:26.466 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:23:26.466 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:23:26.466 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:23:26.466 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:23:26.466 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:23:26.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:23:26.897 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:23:26.898 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:23:26.899 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:23:26.899 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:23:26.899 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:23:26.899 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:26.899 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:27.077 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:14 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:27.217 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:27.218 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:23:27.218 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:27.218 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T08:23:27.218 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:23:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:27.220 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:27.220 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:27.220 INFO:teuthology.orchestra.run.smithi043.stdout:Install 13 Packages 2024-09-11T08:23:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:27.221 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.2 M 2024-09-11T08:23:27.221 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-09-11T08:23:27.221 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:27.250 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:23:27.754 INFO:teuthology.orchestra.run.smithi043.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 258 kB/s | 26 kB 00:00 2024-09-11T08:23:27.779 INFO:teuthology.orchestra.run.smithi043.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 154 kB/s | 19 kB 00:00 2024-09-11T08:23:27.829 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:22 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:27.837 INFO:teuthology.orchestra.run.smithi043.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 259 kB/s | 47 kB 00:00 2024-09-11T08:23:27.871 INFO:teuthology.orchestra.run.smithi043.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-11T08:23:27.913 INFO:teuthology.orchestra.run.smithi043.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-11T08:23:27.938 INFO:teuthology.orchestra.run.smithi043.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 804 kB/s | 54 kB 00:00 2024-09-11T08:23:27.965 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:23:27.966 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout:Install 13 Packages 2024-09-11T08:23:27.967 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:27.968 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 2.2 M 2024-09-11T08:23:27.968 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 26 M 2024-09-11T08:23:27.968 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:27.980 INFO:teuthology.orchestra.run.smithi043.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 483 kB/s | 32 kB 00:00 2024-09-11T08:23:28.006 INFO:teuthology.orchestra.run.smithi043.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-11T08:23:28.031 INFO:teuthology.orchestra.run.smithi043.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-11T08:23:28.065 INFO:teuthology.orchestra.run.smithi043.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-11T08:23:28.082 INFO:teuthology.orchestra.run.smithi043.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-11T08:23:28.107 INFO:teuthology.orchestra.run.smithi043.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 2.2 MB/s | 90 kB 00:00 2024-09-11T08:23:28.107 INFO:teuthology.orchestra.run.smithi172.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 300 kB/s | 26 kB 00:00 2024-09-11T08:23:28.132 INFO:teuthology.orchestra.run.smithi172.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 173 kB/s | 19 kB 00:00 2024-09-11T08:23:28.149 INFO:teuthology.orchestra.run.smithi043.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-11T08:23:28.152 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:28.152 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-11T08:23:28.207 INFO:teuthology.orchestra.run.smithi172.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.9 MB/s | 222 kB 00:00 2024-09-11T08:23:28.223 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:28.232 INFO:teuthology.orchestra.run.smithi172.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-09-11T08:23:28.240 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:28.240 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:28.257 INFO:teuthology.orchestra.run.smithi172.stdout:(5/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 201 kB/s | 47 kB 00:00 2024-09-11T08:23:28.283 INFO:teuthology.orchestra.run.smithi172.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-11T08:23:28.308 INFO:teuthology.orchestra.run.smithi172.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-11T08:23:28.336 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:28.338 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:28.358 INFO:teuthology.orchestra.run.smithi172.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-11T08:23:28.384 INFO:teuthology.orchestra.run.smithi172.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-11T08:23:28.409 INFO:teuthology.orchestra.run.smithi172.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T08:23:28.434 INFO:teuthology.orchestra.run.smithi172.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-11T08:23:28.459 INFO:teuthology.orchestra.run.smithi172.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-11T08:23:28.493 INFO:teuthology.orchestra.run.smithi172.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-11T08:23:28.496 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:28.496 INFO:teuthology.orchestra.run.smithi172.stdout:Total 4.2 MB/s | 2.2 MB 00:00 2024-09-11T08:23:28.561 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:28.577 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:28.577 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:28.664 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:28.665 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:28.863 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:29.038 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:23:29.118 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:23:29.179 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:29.182 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:23:29.218 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:23:29.260 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:23:29.359 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:23:29.366 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:23:29.420 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:23:29.476 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:23:29.486 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:23:29.512 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:23:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:23:29.554 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:23:29.657 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:23:29.774 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:23:29.830 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:23:30.305 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:23:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:23:30.392 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:23:30.464 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:23:30.486 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:23:30.555 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:23:30.611 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:23:30.653 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:23:30.699 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:23:30.722 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:23:31.317 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:23:31.317 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T08:23:31.317 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:23:31.317 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:23:31.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:23:31.542 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:23:31.542 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T08:23:31.542 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:23:31.542 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:23:31.542 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:23:31.543 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:23:31.543 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:23:31.543 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:23:31.543 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:23:31.543 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:23:31.543 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:23:31.543 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:23:31.543 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:23:31.657 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:23:31.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:23:31.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:23:31.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:23:31.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:31.659 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:23:31.873 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:31.874 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:31.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:23:32.134 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:23:32.520 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:19 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:32.661 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:32.662 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:23:32.662 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:32.662 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-09-11T08:23:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:32.664 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 7.0 M 2024-09-11T08:23:32.664 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 33 M 2024-09-11T08:23:32.664 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:32.703 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:27 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:32.836 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:32.837 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:32.837 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:23:32.837 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:32.837 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:32.837 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T08:23:32.837 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:23:32.837 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:23:32.837 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:23:32.838 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:23:32.838 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:23:32.838 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:23:32.838 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:32.838 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:32.838 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:32.838 INFO:teuthology.orchestra.run.smithi172.stdout:Install 6 Packages 2024-09-11T08:23:32.838 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:32.839 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 7.0 M 2024-09-11T08:23:32.839 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 33 M 2024-09-11T08:23:32.839 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:33.001 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.5 MB/s | 248 kB 00:00 2024-09-11T08:23:33.051 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-11T08:23:33.076 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 531 kB/s | 124 kB 00:00 2024-09-11T08:23:33.102 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T08:23:33.144 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-11T08:23:33.211 INFO:teuthology.orchestra.run.smithi172.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 548 kB/s | 124 kB 00:00 2024-09-11T08:23:33.227 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-11T08:23:33.230 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:33.230 INFO:teuthology.orchestra.run.smithi043.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-11T08:23:33.236 INFO:teuthology.orchestra.run.smithi172.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 988 kB/s | 248 kB 00:00 2024-09-11T08:23:33.279 INFO:teuthology.orchestra.run.smithi172.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.3 MB/s | 58 kB 00:00 2024-09-11T08:23:33.303 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:33.304 INFO:teuthology.orchestra.run.smithi172.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.2 MB/s | 74 kB 00:00 2024-09-11T08:23:33.315 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:33.315 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:33.346 INFO:teuthology.orchestra.run.smithi172.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.0 MB/s | 548 kB 00:00 2024-09-11T08:23:33.420 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:33.422 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:33.454 INFO:teuthology.orchestra.run.smithi172.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-11T08:23:33.456 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:33.456 INFO:teuthology.orchestra.run.smithi172.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-09-11T08:23:33.527 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:33.538 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:33.538 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:33.638 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:33.639 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:33.730 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:23:33.951 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:23:34.129 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:23:34.181 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:23:34.544 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:23:34.635 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:23:34.698 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:23:34.713 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:23:34.725 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:23:34.801 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:23:34.856 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:23:34.885 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:23:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:23:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T08:23:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:23:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:23:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:23:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:23:35.708 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:23:35.709 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T08:23:35.709 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:23:35.709 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:23:35.709 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:23:35.709 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:23:35.805 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:23:35.805 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:35.805 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:35.805 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:35.805 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:23:35.805 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:23:35.806 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:23:35.806 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:23:35.806 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:23:35.806 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:35.806 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:35.979 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:35.980 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:36.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2024-09-11T08:23:36.217 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-fuse 2024-09-11T08:23:36.640 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:23 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:36.778 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:36.779 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:23:36.779 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:36.779 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T08:23:36.779 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T08:23:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:23:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:36.780 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:36.780 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:36.780 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-09-11T08:23:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:36.780 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 893 k 2024-09-11T08:23:36.780 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 2.6 M 2024-09-11T08:23:36.781 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:36.789 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:36.923 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:36.924 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:36.924 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:23:36.924 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:36.924 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:36.925 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T08:23:36.925 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T08:23:36.925 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:23:36.925 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:36.925 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:36.926 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:36.926 INFO:teuthology.orchestra.run.smithi172.stdout:Install 2 Packages 2024-09-11T08:23:36.926 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:36.926 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 893 k 2024-09-11T08:23:36.926 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 2.6 M 2024-09-11T08:23:36.926 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:36.937 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 585 kB/s | 80 kB 00:00 2024-09-11T08:23:37.073 INFO:teuthology.orchestra.run.smithi172.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 800 kB/s | 80 kB 00:00 2024-09-11T08:23:37.514 INFO:teuthology.orchestra.run.smithi172.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 1.5 MB/s | 813 kB 00:00 2024-09-11T08:23:37.515 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:37.515 INFO:teuthology.orchestra.run.smithi172.stdout:Total 1.5 MB/s | 893 kB 00:00 2024-09-11T08:23:37.527 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:37.549 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:37.569 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:37.570 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 1.1 MB/s | 813 kB 00:00 2024-09-11T08:23:37.570 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:37.570 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.1 MB/s | 893 kB 00:00 2024-09-11T08:23:37.570 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:37.591 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:37.592 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:37.633 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:37.633 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:37.682 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:37.682 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:37.822 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:37.883 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:37.951 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:23:38.022 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:23:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:23:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:23:38.720 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:23:38.720 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T08:23:38.820 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:23:38.820 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T08:23:39.010 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:23:39.010 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:39.010 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:39.011 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:23:39.011 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:39.011 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:39.107 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:23:39.107 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:39.107 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:39.108 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:23:39.108 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:39.108 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:39.288 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-volume 2024-09-11T08:23:39.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-volume 2024-09-11T08:23:39.871 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:40.003 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:27 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:40.007 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:40.007 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:40.007 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T08:23:40.007 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:40.007 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:40.007 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T08:23:40.007 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:40.008 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:40.008 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:40.008 INFO:teuthology.orchestra.run.smithi172.stdout:Install 1 Package 2024-09-11T08:23:40.008 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:40.008 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 270 k 2024-09-11T08:23:40.008 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 1.3 M 2024-09-11T08:23:40.009 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:40.141 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:40.141 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 270 k 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 1.3 M 2024-09-11T08:23:40.142 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:40.219 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.3 MB/s | 270 kB 00:00 2024-09-11T08:23:40.219 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:40.220 INFO:teuthology.orchestra.run.smithi172.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-11T08:23:40.220 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:40.228 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:40.228 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:40.257 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:40.257 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.3 MB/s | 270 kB 00:00 2024-09-11T08:23:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.3 MB/s | 270 kB 00:00 2024-09-11T08:23:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:40.469 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:40.537 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:40.553 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:40.553 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:23:40.553 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:23:40.553 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:40.670 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:40.688 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:40.688 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:23:40.688 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:23:40.688 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:41.547 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:41.547 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:41.547 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:41.547 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:41.547 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:41.547 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:23:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:41.814 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:23:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:41.814 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:41.832 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install librados-devel 2024-09-11T08:23:42.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados-devel 2024-09-11T08:23:42.406 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:42.543 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repo Size 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:42.544 INFO:teuthology.orchestra.run.smithi172.stdout:Install 1 Package 2024-09-11T08:23:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:42.545 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 126 k 2024-09-11T08:23:42.545 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 448 k 2024-09-11T08:23:42.545 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:42.700 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:42.740 INFO:teuthology.orchestra.run.smithi172.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 646 kB/s | 126 kB 00:00 2024-09-11T08:23:42.740 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:42.741 INFO:teuthology.orchestra.run.smithi172.stdout:Total 642 kB/s | 126 kB 00:00 2024-09-11T08:23:42.741 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:42.746 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:42.746 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:42.810 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:42.810 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:42.840 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:42.841 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:42.841 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-11T08:23:42.841 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 126 k 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 448 k 2024-09-11T08:23:42.842 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:43.056 INFO:teuthology.orchestra.run.smithi043.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 590 kB/s | 126 kB 00:00 2024-09-11T08:23:43.057 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:43.057 INFO:teuthology.orchestra.run.smithi043.stdout:Total 586 kB/s | 126 kB 00:00 2024-09-11T08:23:43.057 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:43.063 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:43.063 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:43.288 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:43.399 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:43.753 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:43.753 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:43.832 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:44.025 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install libcephfs2 2024-09-11T08:23:44.081 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:44.081 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:44.082 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:44.082 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:44.082 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:44.082 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:44.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2024-09-11T08:23:44.607 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:44.699 INFO:teuthology.orchestra.run.smithi172.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:44.743 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:44.744 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:23:44.744 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:44.805 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install libcephfs-devel 2024-09-11T08:23:44.966 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:45.058 INFO:teuthology.orchestra.run.smithi043.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:45.103 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:45.105 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:45.105 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:45.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2024-09-11T08:23:45.367 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:45.501 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:45.501 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repo Size 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout:Install 1 Package 2024-09-11T08:23:45.502 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:45.503 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 25 k 2024-09-11T08:23:45.503 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 110 k 2024-09-11T08:23:45.503 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:45.652 INFO:teuthology.orchestra.run.smithi172.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 167 kB/s | 25 kB 00:00 2024-09-11T08:23:45.653 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:45.653 INFO:teuthology.orchestra.run.smithi172.stdout:Total 166 kB/s | 25 kB 00:00 2024-09-11T08:23:45.653 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:45.656 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:45.657 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:45.658 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:45.658 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:45.744 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:45.757 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:45.859 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:45.896 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:45.897 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:45.897 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-11T08:23:45.897 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:45.897 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:45.897 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:23:45.897 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:45.897 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:45.897 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:45.899 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-11T08:23:45.900 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:45.900 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 25 k 2024-09-11T08:23:45.900 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 110 k 2024-09-11T08:23:45.900 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:46.052 INFO:teuthology.orchestra.run.smithi043.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 162 kB/s | 25 kB 00:00 2024-09-11T08:23:46.052 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:46.053 INFO:teuthology.orchestra.run.smithi043.stdout:Total 161 kB/s | 25 kB 00:00 2024-09-11T08:23:46.053 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:46.056 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:46.056 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:46.057 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:46.058 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:46.134 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:46.225 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:46.252 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:46.470 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:46.470 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:46.470 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:46.471 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:46.471 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:46.471 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:46.645 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install librados2 2024-09-11T08:23:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:46.886 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:46.886 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:47.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2024-09-11T08:23:47.210 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:47.303 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:47.345 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:47.346 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:23:47.346 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:47.406 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install librbd1 2024-09-11T08:23:47.648 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:47.741 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:47.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2024-09-11T08:23:47.968 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:48.059 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:48.102 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:48.103 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:23:48.103 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:48.160 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install python3-rados 2024-09-11T08:23:48.435 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:48.529 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:48.574 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:48.576 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:48.576 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:48.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rados 2024-09-11T08:23:48.722 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:48.811 INFO:teuthology.orchestra.run.smithi172.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:48.854 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:48.855 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:23:48.856 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:48.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install python3-rgw 2024-09-11T08:23:49.239 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:49.331 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:49.377 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:49.378 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:49.378 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:49.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rgw 2024-09-11T08:23:49.479 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:44 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:49.568 INFO:teuthology.orchestra.run.smithi172.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:49.611 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:49.612 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:23:49.612 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:49.669 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install python3-cephfs 2024-09-11T08:23:50.030 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:50.121 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:50.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-cephfs 2024-09-11T08:23:50.230 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:50.321 INFO:teuthology.orchestra.run.smithi172.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:50.365 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:50.366 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:23:50.366 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:50.427 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install python3-rbd 2024-09-11T08:23:50.821 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:50.915 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:50.960 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:50.961 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:50.961 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:50.986 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:51.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rbd 2024-09-11T08:23:51.076 INFO:teuthology.orchestra.run.smithi172.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:51.118 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:51.119 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-09-11T08:23:51.119 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:51.178 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install rbd-fuse 2024-09-11T08:23:51.615 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:51.708 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:23:51.739 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:46 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:51.753 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:51.754 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-11T08:23:51.754 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:51.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2024-09-11T08:23:51.872 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:51.873 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:51.873 INFO:teuthology.orchestra.run.smithi172.stdout: Package Architecture Version Repository Size 2024-09-11T08:23:51.873 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:51.873 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:51.873 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T08:23:51.874 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:51.874 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:51.874 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:51.874 INFO:teuthology.orchestra.run.smithi172.stdout:Install 1 Package 2024-09-11T08:23:51.874 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:51.874 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 82 k 2024-09-11T08:23:51.874 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 222 k 2024-09-11T08:23:51.874 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:52.072 INFO:teuthology.orchestra.run.smithi172.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 414 kB/s | 82 kB 00:00 2024-09-11T08:23:52.072 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:52.073 INFO:teuthology.orchestra.run.smithi172.stdout:Total 411 kB/s | 82 kB 00:00 2024-09-11T08:23:52.073 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:52.079 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:52.079 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:52.143 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:52.143 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:52.283 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:52.393 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:52.405 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:52.545 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:52.545 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:52.545 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-11T08:23:52.545 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 82 k 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 222 k 2024-09-11T08:23:52.546 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:52.755 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:52.838 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 280 kB/s | 82 kB 00:00 2024-09-11T08:23:52.839 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:52.839 INFO:teuthology.orchestra.run.smithi043.stdout:Total 279 kB/s | 82 kB 00:00 2024-09-11T08:23:52.839 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:52.847 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:52.847 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:52.915 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:52.922 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:53.011 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:53.011 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:53.011 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:53.012 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:53.012 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:53.012 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:53.070 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:53.239 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:53.253 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install rbd-mirror 2024-09-11T08:23:53.633 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:53.826 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:48 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:53.850 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:53.850 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:53.851 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:53.851 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:53.851 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:53.851 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:53.964 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:53.964 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repo Size 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:Install 1 Package 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 2.9 M 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 12 M 2024-09-11T08:23:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:54.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2024-09-11T08:23:54.291 INFO:teuthology.orchestra.run.smithi172.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 9.0 MB/s | 2.9 MB 00:00 2024-09-11T08:23:54.292 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:54.292 INFO:teuthology.orchestra.run.smithi172.stdout:Total 9.0 MB/s | 2.9 MB 00:00 2024-09-11T08:23:54.293 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:54.301 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:54.301 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:54.383 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:54.384 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:54.683 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:54.817 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:54.823 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-11T08:23:54.824 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:54.825 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.9 M 2024-09-11T08:23:54.825 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 12 M 2024-09-11T08:23:54.826 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:54.876 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:54.903 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:54.903 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:23:54.904 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:23:54.904 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:23:54.904 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:23:54.904 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:55.168 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.6 MB/s | 2.9 MB 00:00 2024-09-11T08:23:55.169 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:55.169 INFO:teuthology.orchestra.run.smithi043.stdout:Total 8.5 MB/s | 2.9 MB 00:00 2024-09-11T08:23:55.169 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:55.179 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:55.179 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:55.263 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:55.264 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:55.696 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:55.768 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:55.795 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:55.795 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:23:55.795 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:23:55.795 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:23:55.795 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:23:55.795 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:56.084 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:56.084 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:56.084 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:56.084 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:56.084 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:56.085 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:56.320 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install rbd-nbd 2024-09-11T08:23:56.906 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:51 ago on Wed 11 Sep 2024 08:22:05 AM UTC. 2024-09-11T08:23:56.986 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:56.986 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:56.986 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:56.986 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:56.987 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:56.987 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:57.041 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T08:23:57.041 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:57.041 INFO:teuthology.orchestra.run.smithi172.stdout: Package Architecture Version Repository Size 2024-09-11T08:23:57.041 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout:Install 1 Package 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 165 k 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 477 k 2024-09-11T08:23:57.042 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T08:23:57.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-09-11T08:23:57.254 INFO:teuthology.orchestra.run.smithi172.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 782 kB/s | 165 kB 00:00 2024-09-11T08:23:57.254 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:57.255 INFO:teuthology.orchestra.run.smithi172.stdout:Total 777 kB/s | 165 kB 00:00 2024-09-11T08:23:57.255 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T08:23:57.262 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T08:23:57.262 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T08:23:57.327 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T08:23:57.327 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T08:23:57.476 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T08:23:57.584 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:57.827 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:44 ago on Wed 11 Sep 2024 08:22:13 AM UTC. 2024-09-11T08:23:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T08:23:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-11T08:23:57.967 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:57.968 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 165 k 2024-09-11T08:23:57.968 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 477 k 2024-09-11T08:23:57.968 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T08:23:57.991 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:58.187 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 754 kB/s | 165 kB 00:00 2024-09-11T08:23:58.188 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:58.188 INFO:teuthology.orchestra.run.smithi043.stdout:Total 748 kB/s | 165 kB 00:00 2024-09-11T08:23:58.188 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T08:23:58.196 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T08:23:58.197 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T08:23:58.248 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:58.249 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:58.249 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T08:23:58.249 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:58.249 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:23:58.249 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T08:23:58.264 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T08:23:58.265 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T08:23:58.429 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T08:23:58.492 DEBUG:teuthology.parallel:result is None 2024-09-11T08:23:58.539 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:58.964 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:59.218 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:23:59.218 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:59.218 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T08:23:59.218 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:23:59.219 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:23:59.219 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T08:23:59.459 DEBUG:teuthology.parallel:result is None 2024-09-11T08:23:59.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:23:59.611 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:23:59.634 INFO:teuthology.orchestra.run.smithi043.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T08:23:59.634 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T08:23:59.634 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T08:23:59.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:23:59.782 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:23:59.808 INFO:teuthology.orchestra.run.smithi172.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T08:23:59.808 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T08:23:59.808 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T08:23:59.810 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T08:23:59.810 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:23:59.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:23:59.838 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:23:59.838 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:23:59.877 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T08:23:59.877 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:23:59.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:23:59.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:23:59.971 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:23:59.972 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:24:00.000 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:24:00.068 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T08:24:00.069 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:24:00.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:24:00.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:24:00.162 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:24:00.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:24:00.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:24:00.255 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T08:24:00.256 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:24:00.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:24:00.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:24:00.347 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:24:00.347 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:24:00.374 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:24:00.440 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T08:24:00.550 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'cephadm_mode': 'cephadm-package'} 2024-09-11T08:24:00.550 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:24:00.551 INFO:tasks.cephadm:Cluster fsid is 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:24:00.551 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T08:24:00.551 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.c': '[v2:172.21.15.43:3301,v1:172.21.15.43:6790]', 'mon.b': '172.21.15.172'} 2024-09-11T08:24:00.551 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-09-11T08:24:00.551 INFO:tasks.cephadm:First mgr is y 2024-09-11T08:24:00.551 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T08:24:00.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-09-11T08:24:00.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname $(hostname -s) 2024-09-11T08:24:00.616 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T08:24:00.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:24:00.622 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:24:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:24:02.268 INFO:teuthology.orchestra.run.smithi172.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:24:29.919 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-11T08:24:29.920 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:24:29.920 INFO:teuthology.orchestra.run.smithi172.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:24:29.920 INFO:teuthology.orchestra.run.smithi172.stdout: "repo_digests": [ 2024-09-11T08:24:29.920 INFO:teuthology.orchestra.run.smithi172.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:24:29.920 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-11T08:24:29.920 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-11T08:24:34.129 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T08:24:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:24:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:24:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-09-11T08:24:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:24:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T08:24:34.129 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T08:24:34.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-09-11T08:24:34.206 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph 2024-09-11T08:24:34.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-09-11T08:24:34.265 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /etc/ceph 2024-09-11T08:24:34.308 INFO:tasks.cephadm:Writing seed config... 2024-09-11T08:24:34.309 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T08:24:34.310 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-11T08:24:34.311 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-11T08:24:34.311 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T08:24:34.311 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:24:34.311 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T08:24:34.327 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 327ee848-7017-11ef-bcea-c7b262605968 mon election default strategy = 1 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-11T08:24:34.328 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T08:24:34.370 DEBUG:teuthology.orchestra.run.smithi043:mgr.y> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T08:24:34.413 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T08:24:34.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid 327ee848-7017-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:24:34.619 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:24:34.619 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', '327ee848-7017-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-09-11T08:24:34.644 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:24:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T08:24:34.645 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-09-11T08:24:34.668 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:24:34.668 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-09-11T08:24:34.669 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-09-11T08:24:34.677 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:24:34.677 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:24:34.684 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:24:34.684 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T08:24:34.691 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-11T08:24:34.698 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-11T08:24:34.698 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:24:34.698 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-09-11T08:24:34.721 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:24:34.721 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T08:24:34.721 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-09-11T08:24:34.721 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-09-11T08:24:34.728 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:24:34.728 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:24:34.735 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:24:34.735 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T08:24:34.742 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-11T08:24:34.749 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-11T08:24:34.750 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:24:34.750 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-09-11T08:24:34.750 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:24:34.750 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 139944292860688 on /run/cephadm/327ee848-7017-11ef-bcea-c7b262605968.lock 2024-09-11T08:24:34.750 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 139944292860688 acquired on /run/cephadm/327ee848-7017-11ef-bcea-c7b262605968.lock 2024-09-11T08:24:34.751 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-09-11T08:24:34.751 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-09-11T08:24:34.751 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-11T08:24:34.754 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-09-11T08:24:34.755 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-09-11T08:24:34.758 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T08:24:34.758 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-11T08:24:34.758 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-09-11T08:24:34.760 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T08:24:34.761 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T08:24:34.761 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:24:34.761 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-11T08:24:34.761 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-11T08:24:34.761 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:24:34.761 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-11T08:24:34.762 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-11T08:24:34.762 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T08:24:34.762 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T08:24:34.763 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:24:35.254 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:24:35.254 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:24:35.254 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T08:24:35.254 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T08:24:35.254 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T08:24:35.254 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:24:35.254 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T08:24:36.133 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:24:36.133 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:24:36.133 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T08:24:37.030 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-09-11T08:24:37.030 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-09-11T08:24:37.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQDFU+Fm+sWjFRAAww4cgmvQ+JueSM7rAlyYLQ== 2024-09-11T08:24:38.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQDGU+FmRIoiFBAADa0Qz5H8KwWrkmBZ+guUow== 2024-09-11T08:24:39.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQDHU+FmkvrqERAA6Eflzn+bwFt9T3hfOwf0oA== 2024-09-11T08:24:39.865 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-09-11T08:24:40.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:24:40.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T08:24:40.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:24:40.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:24:40.854 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:24:40.854 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = octopus 2024-09-11T08:24:40.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:24:40.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:24:40.855 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:24:40.855 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-09-11T08:24:41.751 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-09-11T08:24:42.043 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T08:24:42.194 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T08:24:42.344 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-327ee848-7017-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-327ee848-7017-11ef-bcea-c7b262605968.target. 2024-09-11T08:24:42.345 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-327ee848-7017-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-327ee848-7017-11ef-bcea-c7b262605968.target. 2024-09-11T08:24:42.523 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:24:42.523 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T08:24:42.696 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-327ee848-7017-11ef-bcea-c7b262605968.target.wants/ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-327ee848-7017-11ef-bcea-c7b262605968@.service. 2024-09-11T08:24:43.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:43 smithi043 ceph-mon[28885]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:24:43.215 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:24:43.216 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-11T08:24:43.223 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:24:43.224 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T08:24:43.224 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-11T08:24:43.224 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:24:43.224 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-09-11T08:24:43.224 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-09-11T08:24:43.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:43 smithi043 bash[28848]: 05a72eded45bd1d717284253f99cfead5ebc68409ae55133a98655fbfb2cf817 2024-09-11T08:24:43.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:43 smithi043 systemd[1]: Started Ceph mon.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:24:44.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:44 smithi043 ceph-mon[28885]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:24:44.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:44 smithi043 ceph-mon[28885]: monmap e1: 1 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:24:44.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:44 smithi043 ceph-mon[28885]: fsmap 2024-09-11T08:24:44.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:44 smithi043 ceph-mon[28885]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:24:44.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:44 smithi043 ceph-mon[28885]: mgrmap e1: no daemons active 2024-09-11T08:24:44.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:44 smithi043 ceph-mon[28885]: from='client.? 172.21.15.43:0/1803401305' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.583146s) 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:24:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.674+0000 7f071b8b2640 1 Processor -- start 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.674+0000 7f071b8b2640 1 -- start start 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.675+0000 7f071b8b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f07141009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.675+0000 7f071b8b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0714100f10 con 0x7f07141005c0 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.675+0000 7f071964e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f07141009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:24:49.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.675+0000 7f071964e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f07141009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34812/0 (socket says 172.21.15.43:34812) 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.675+0000 7f071964e640 1 -- 172.21.15.43:0/3117615821 learned_addr learned my addr 172.21.15.43:0/3117615821 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.675+0000 7f071964e640 1 -- 172.21.15.43:0/3117615821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0714101050 con 0x7f07141005c0 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.677+0000 7f071964e640 1 --2- 172.21.15.43:0/3117615821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f07141009d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f070c009840 tx=0x7f070c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b915d531acfa9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.677+0000 7f070bfff640 1 -- 172.21.15.43:0/3117615821 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f070c02add0 con 0x7f07141005c0 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.677+0000 7f070bfff640 1 -- 172.21.15.43:0/3117615821 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c032440 con 0x7f07141005c0 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.677+0000 7f070bfff640 1 -- 172.21.15.43:0/3117615821 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f070c032750 con 0x7f07141005c0 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.678+0000 7f071b8b2640 1 -- 172.21.15.43:0/3117615821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 msgr2=0x7f07141009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.678+0000 7f071b8b2640 1 --2- 172.21.15.43:0/3117615821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f07141009d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f070c009840 tx=0x7f070c0091d0 comp rx=0 tx=0).stop 2024-09-11T08:24:49.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.678+0000 7f071b8b2640 1 -- 172.21.15.43:0/3117615821 shutdown_connections 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.678+0000 7f071b8b2640 1 --2- 172.21.15.43:0/3117615821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f07141009d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.678+0000 7f071b8b2640 1 -- 172.21.15.43:0/3117615821 >> 172.21.15.43:0/3117615821 conn(0x7f07140fbe10 msgr2=0x7f07140fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.678+0000 7f071b8b2640 1 -- 172.21.15.43:0/3117615821 shutdown_connections 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.678+0000 7f071b8b2640 1 -- 172.21.15.43:0/3117615821 wait complete. 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.679+0000 7f071b8b2640 1 Processor -- start 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.679+0000 7f071b8b2640 1 -- start start 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.679+0000 7f071b8b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f071410ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.679+0000 7f071b8b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071410f010 con 0x7f07141005c0 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.679+0000 7f071964e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f071410ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:24:49.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.679+0000 7f071964e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f071410ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34824/0 (socket says 172.21.15.43:34824) 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.679+0000 7f071964e640 1 -- 172.21.15.43:0/1803401305 learned_addr learned my addr 172.21.15.43:0/1803401305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.679+0000 7f071964e640 1 -- 172.21.15.43:0/1803401305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c009450 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.680+0000 7f071964e640 1 --2- 172.21.15.43:0/1803401305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f071410ead0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f070c02a700 tx=0x7f070c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.680+0000 7f0709ffb640 1 -- 172.21.15.43:0/1803401305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f070c031a80 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.680+0000 7f0709ffb640 1 -- 172.21.15.43:0/1803401305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c031be0 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.680+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071410f210 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.680+0000 7f0709ffb640 1 -- 172.21.15.43:0/1803401305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f070c03b400 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.680+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071410f620 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.680+0000 7f0709ffb640 1 -- 172.21.15.43:0/1803401305 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f070c039040 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.680+0000 7f0709ffb640 1 -- 172.21.15.43:0/1803401305 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f070c041030 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.681+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071404b720 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.683+0000 7f0709ffb640 1 -- 172.21.15.43:0/1803401305 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f070c037070 con 0x7f07141005c0 2024-09-11T08:24:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.748+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f0714101810 con 0x7f07141005c0 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.748+0000 7f0709ffb640 1 -- 172.21.15.43:0/1803401305 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f070c055070 con 0x7f07141005c0 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.749+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 msgr2=0x7f071410ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.749+0000 7f071b8b2640 1 --2- 172.21.15.43:0/1803401305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f071410ead0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f070c02a700 tx=0x7f070c032000 comp rx=0 tx=0).stop 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.749+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 shutdown_connections 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.749+0000 7f071b8b2640 1 --2- 172.21.15.43:0/1803401305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07141005c0 0x7f071410ead0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.749+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 >> 172.21.15.43:0/1803401305 conn(0x7f07140fbe10 msgr2=0x7f0714102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.749+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 shutdown_connections 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:43.749+0000 7f071b8b2640 1 -- 172.21.15.43:0/1803401305 wait complete. 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-09-11T08:24:49.907 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T08:24:53.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:53 smithi043 ceph-mon[28885]: from='client.? 172.21.15.43:0/3736765621' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:24:53.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:53 smithi043 ceph-mon[28885]: from='client.? 172.21.15.43:0/3736765621' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T08:24:55.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:24:55.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:24:55.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:24:55.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.992+0000 7f6195752640 1 Processor -- start 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.992+0000 7f6195752640 1 -- start start 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.992+0000 7f6195752640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190100960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.992+0000 7f6195752640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6190100f30 con 0x7f6190100550 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.993+0000 7f618effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.993+0000 7f618effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190100960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55956/0 (socket says 172.21.15.43:55956) 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.993+0000 7f618effd640 1 -- 172.21.15.43:0/798787810 learned_addr learned my addr 172.21.15.43:0/798787810 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.993+0000 7f618effd640 1 -- 172.21.15.43:0/798787810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6190101780 con 0x7f6190100550 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f618effd640 1 --2- 172.21.15.43:0/798787810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190100960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6180009840 tx=0x7f61800091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d03fb1f8ca96dac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f618dffb640 1 -- 172.21.15.43:0/798787810 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618002add0 con 0x7f6190100550 2024-09-11T08:24:55.807 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f618dffb640 1 -- 172.21.15.43:0/798787810 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6180032440 con 0x7f6190100550 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f618dffb640 1 -- 172.21.15.43:0/798787810 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6180032750 con 0x7f6190100550 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f6195752640 1 -- 172.21.15.43:0/798787810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 msgr2=0x7f6190100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f6195752640 1 --2- 172.21.15.43:0/798787810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190100960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6180009840 tx=0x7f61800091d0 comp rx=0 tx=0).stop 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f6195752640 1 -- 172.21.15.43:0/798787810 shutdown_connections 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f6195752640 1 --2- 172.21.15.43:0/798787810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190100960 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f6195752640 1 -- 172.21.15.43:0/798787810 >> 172.21.15.43:0/798787810 conn(0x7f61900fbc50 msgr2=0x7f61900fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f6195752640 1 -- 172.21.15.43:0/798787810 shutdown_connections 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.994+0000 7f6195752640 1 -- 172.21.15.43:0/798787810 wait complete. 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.995+0000 7f6195752640 1 Processor -- start 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.995+0000 7f6195752640 1 -- start start 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.995+0000 7f6195752640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.995+0000 7f6195752640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6190077160 con 0x7f6190100550 2024-09-11T08:24:55.808 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.995+0000 7f618effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f618effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190076c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55958/0 (socket says 172.21.15.43:55958) 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f618effd640 1 -- 172.21.15.43:0/3736765621 learned_addr learned my addr 172.21.15.43:0/3736765621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f618effd640 1 -- 172.21.15.43:0/3736765621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180009450 con 0x7f6190100550 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f618effd640 1 --2- 172.21.15.43:0/3736765621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190076c20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f618002a700 tx=0x7f6180032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f61877fe640 1 -- 172.21.15.43:0/3736765621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6180031a80 con 0x7f6190100550 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f61877fe640 1 -- 172.21.15.43:0/3736765621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6180031be0 con 0x7f6190100550 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61900751f0 con 0x7f6190100550 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f61877fe640 1 -- 172.21.15.43:0/3736765621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618003b400 con 0x7f6190100550 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6190075600 con 0x7f6190100550 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f61877fe640 1 -- 172.21.15.43:0/3736765621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6180039040 con 0x7f6190100550 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.996+0000 7f61877fe640 1 -- 172.21.15.43:0/3736765621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6180041030 con 0x7f6190100550 2024-09-11T08:24:55.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.997+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61900760c0 con 0x7f6190100550 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:52.999+0000 7f61877fe640 1 -- 172.21.15.43:0/3736765621 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f61900760c0 con 0x7f6190100550 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.057+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6190101c70 con 0x7f6190100550 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.060+0000 7f61877fe640 1 -- 172.21.15.43:0/3736765621 <== mon.0 v2:172.21.15.43:3300/0 7 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f618004f3e0 con 0x7f6190100550 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.060+0000 7f61877fe640 1 -- 172.21.15.43:0/3736765621 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+374 (secure 0 0 0) 0x7f6180055020 con 0x7f6190100550 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.061+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 msgr2=0x7f6190076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.061+0000 7f6195752640 1 --2- 172.21.15.43:0/3736765621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190076c20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f618002a700 tx=0x7f6180032000 comp rx=0 tx=0).stop 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.061+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 shutdown_connections 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.061+0000 7f6195752640 1 --2- 172.21.15.43:0/3736765621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6190100550 0x7f6190076c20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.061+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 >> 172.21.15.43:0/3736765621 conn(0x7f61900fbc50 msgr2=0x7f6190105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.061+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 shutdown_connections 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:53.061+0000 7f6195752640 1 -- 172.21.15.43:0/3736765621 wait complete. 2024-09-11T08:24:55.810 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-09-11T08:24:56.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:56 smithi043 ceph-mon[28885]: from='client.? 172.21.15.43:0/2182072590' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:24:56.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.215+0000 7f1f70f82640 1 Processor -- start 2024-09-11T08:24:56.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.215+0000 7f1f70f82640 1 -- start start 2024-09-11T08:24:56.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.215+0000 7f1f70f82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.215+0000 7f1f70f82640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c100f00 con 0x7f1f6c1005b0 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.215+0000 7f1f6a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.216+0000 7f1f6a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c1009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55974/0 (socket says 172.21.15.43:55974) 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.216+0000 7f1f6a59c640 1 -- 172.21.15.43:0/1698748746 learned_addr learned my addr 172.21.15.43:0/1698748746 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.216+0000 7f1f6a59c640 1 -- 172.21.15.43:0/1698748746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c101040 con 0x7f1f6c1005b0 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.217+0000 7f1f6a59c640 1 --2- 172.21.15.43:0/1698748746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c1009c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009840 tx=0x7f1f5c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dab1ffd2513e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.217+0000 7f1f6959a640 1 -- 172.21.15.43:0/1698748746 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f5c02add0 con 0x7f1f6c1005b0 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.217+0000 7f1f6959a640 1 -- 172.21.15.43:0/1698748746 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f1f5c032890 con 0x7f1f6c1005b0 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.217+0000 7f1f6959a640 1 -- 172.21.15.43:0/1698748746 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f5c0317a0 con 0x7f1f6c1005b0 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.217+0000 7f1f70f82640 1 -- 172.21.15.43:0/1698748746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 msgr2=0x7f1f6c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.217+0000 7f1f70f82640 1 --2- 172.21.15.43:0/1698748746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c1009c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009840 tx=0x7f1f5c0091d0 comp rx=0 tx=0).stop 2024-09-11T08:24:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 -- 172.21.15.43:0/1698748746 shutdown_connections 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 --2- 172.21.15.43:0/1698748746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c1009c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 -- 172.21.15.43:0/1698748746 >> 172.21.15.43:0/1698748746 conn(0x7f1f6c0fbe10 msgr2=0x7f1f6c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 -- 172.21.15.43:0/1698748746 shutdown_connections 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 -- 172.21.15.43:0/1698748746 wait complete. 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 Processor -- start 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 -- start start 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c10f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f70f82640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c10f780 con 0x7f1f6c1005b0 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f6a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c10f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f6a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c10f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55978/0 (socket says 172.21.15.43:55978) 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.218+0000 7f1f6a59c640 1 -- 172.21.15.43:0/2182072590 learned_addr learned my addr 172.21.15.43:0/2182072590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:24:56.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f6a59c640 1 -- 172.21.15.43:0/2182072590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009450 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f6a59c640 1 --2- 172.21.15.43:0/2182072590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c10f240 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0314e0 tx=0x7f1f5c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f5affd640 1 -- 172.21.15.43:0/2182072590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f5c03c400 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f5affd640 1 -- 172.21.15.43:0/2182072590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f1f5c03c9b0 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f5affd640 1 -- 172.21.15.43:0/2182072590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f5c03a530 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c10f980 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c10fdf0 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f5affd640 1 -- 172.21.15.43:0/2182072590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1f5c03aa80 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.219+0000 7f1f5affd640 1 -- 172.21.15.43:0/2182072590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1f5c046c40 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.220+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c04b720 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.222+0000 7f1f5affd640 1 -- 172.21.15.43:0/2182072590 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f1f6c10fdf0 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.280+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1f6c1105b0 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.281+0000 7f1f5affd640 1 -- 172.21.15.43:0/2182072590 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f1f5c037070 con 0x7f1f6c1005b0 2024-09-11T08:24:56.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.281+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 msgr2=0x7f1f6c10f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:24:56.830 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.281+0000 7f1f70f82640 1 --2- 172.21.15.43:0/2182072590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c10f240 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0314e0 tx=0x7f1f5c0091d0 comp rx=0 tx=0).stop 2024-09-11T08:24:56.830 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.281+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 shutdown_connections 2024-09-11T08:24:56.830 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.281+0000 7f1f70f82640 1 --2- 172.21.15.43:0/2182072590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6c1005b0 0x7f1f6c10f240 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:24:56.830 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.281+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 >> 172.21.15.43:0/2182072590 conn(0x7f1f6c0fbe10 msgr2=0x7f1f6c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:24:56.830 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.282+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 shutdown_connections 2024-09-11T08:24:56.830 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:56.282+0000 7f1f70f82640 1 -- 172.21.15.43:0/2182072590 wait complete. 2024-09-11T08:24:56.830 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-09-11T08:24:57.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:56 smithi043 systemd[1]: Stopping Ceph mon.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:24:57.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a[28862]: 2024-09-11T08:24:57.099+0000 7f3aa820f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:24:57.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a[28862]: 2024-09-11T08:24:57.099+0000 7f3aa820f640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T08:24:57.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 podman[29199]: 2024-09-11 08:24:57.192122419 +0000 UTC m=+0.201223902 container died 05a72eded45bd1d717284253f99cfead5ebc68409ae55133a98655fbfb2cf817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T08:24:57.831 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 podman[29199]: 2024-09-11 08:24:57.54753979 +0000 UTC m=+0.556641321 container cleanup 05a72eded45bd1d717284253f99cfead5ebc68409ae55133a98655fbfb2cf817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:24:57.831 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 bash[29199]: ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:24:58.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 podman[29216]: 2024-09-11 08:24:57.831631667 +0000 UTC m=+0.630082343 container remove 05a72eded45bd1d717284253f99cfead5ebc68409ae55133a98655fbfb2cf817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:24:58.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T08:24:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 systemd[1]: Stopped Ceph mon.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:24:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:57 smithi043 systemd[1]: Starting Ceph mon.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:24:58.467 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 podman[29317]: 2024-09-11 08:24:58.092113179 +0000 UTC m=+0.016176741 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:24:58.467 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 podman[29317]: 2024-09-11 08:24:58.199385934 +0000 UTC m=+0.123449528 container create b5c5c813fd92f986ab7255741249cf21770b7002ccd026747e659a711b55a953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 podman[29317]: 2024-09-11 08:24:58.38498736 +0000 UTC m=+0.309050913 container init b5c5c813fd92f986ab7255741249cf21770b7002ccd026747e659a711b55a953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True) 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 podman[29317]: 2024-09-11 08:24:58.389694757 +0000 UTC m=+0.313758310 container start b5c5c813fd92f986ab7255741249cf21770b7002ccd026747e659a711b55a953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: pidfile_write: ignore empty --pid-file 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: load: jerasure load: lrc 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:24:58.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: DB SUMMARY 2024-09-11T08:24:58.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: DB Session ID: ODVFG23NTH168L9I9QTK 2024-09-11T08:24:58.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:24:58.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:24:58.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T08:24:58.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T08:24:58.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 85934 ; 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.env: 0x5628ad424440 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.fs: Posix File System 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.info_log: 0x5628adebcec0 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:24:58.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.statistics: (nil) 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.use_fsync: 0 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:24:58.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.db_log_dir: 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.write_buffer_manager: 0x5628adf7a7e0 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:24:58.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.unordered_write: 0 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.row_cache: None 2024-09-11T08:24:58.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.wal_filter: None 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:24:58.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:24:58.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_open_files: -1 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Compression algorithms supported: 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: kZSTD supported: 0 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:24:58.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.merge_operator: 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_filter: None 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5628ade63a38) 2024-09-11T08:24:58.479 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: hash_index_allow_collision: 1 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 1 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x5628ade991f0 2024-09-11T08:24:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-09-11T08:24:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-09-11T08:24:58.482 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-09-11T08:24:58.482 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-09-11T08:24:58.482 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:24:58.482 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-09-11T08:24:58.482 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-09-11T08:24:58.483 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:24:58.483 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 4 2024-09-11T08:24:58.483 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-09-11T08:24:58.483 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-09-11T08:24:58.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:24:58.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:24:58.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compression: NoCompression 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.num_levels: 7 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:24:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:24:58.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:24:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:24:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.ttl: 2592000 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:24:58.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043098413946, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043098414787, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 82236, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 80482, "index_size": 216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10546, "raw_average_key_size": 47, "raw_value_size": 74586, "raw_average_value_size": 337, "num_data_blocks": 10, "num_entries": 221, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726043098, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "9b0574c4-f8f1-4a63-ac43-65ace7e0de2d", "db_session_id": "ODVFG23NTH168L9I9QTK"}} 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043098415156, "job": 1, "event": "recovery_finished"} 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5628adecc700 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: DB pointer 0x5628ade9c000 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-09-11T08:24:58.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 82.09 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.5 0.00 0.00 1 0.001 0 0 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 82.09 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.5 0.00 0.00 1 0.001 0 0 2024-09-11T08:24:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.5 0.00 0.00 1 0.001 0 0 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 139.5 0.00 0.00 1 0.001 0 0 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 23.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:24:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 23.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 82.09 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.5 0.00 0.00 1 0.001 0 0 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 82.09 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.5 0.00 0.00 1 0.001 0 0 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:24:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 139.5 0.00 0.00 1 0.001 0 0 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 23.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:24:58.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a@-1(???) e1 preinit fsid 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a@-1(???).mds e1 new map 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a@-1(???).mds e1 print_map 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 bash[29317]: b5c5c813fd92f986ab7255741249cf21770b7002ccd026747e659a711b55a953 2024-09-11T08:24:58.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 systemd[1]: Started Ceph mon.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:24:58.496 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T08:24:58.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:24:58.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: monmap e1: 1 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:24:58.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: fsmap 2024-09-11T08:24:58.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:24:58.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:58 smithi043 ceph-mon[29355]: mgrmap e1: no daemons active 2024-09-11T08:24:59.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.877+0000 7fa165fcc640 1 Processor -- start 2024-09-11T08:24:59.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.877+0000 7fa165fcc640 1 -- start start 2024-09-11T08:24:59.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.877+0000 7fa165fcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa160100a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:24:59.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.877+0000 7fa165fcc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa160100fc0 con 0x7fa160100670 2024-09-11T08:24:59.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.878+0000 7fa15f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa160100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:24:59.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.878+0000 7fa15f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa160100a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55980/0 (socket says 172.21.15.43:55980) 2024-09-11T08:24:59.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.878+0000 7fa15f7fe640 1 -- 172.21.15.43:0/4189304947 learned_addr learned my addr 172.21.15.43:0/4189304947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:24:59.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.878+0000 7fa15f7fe640 1 -- 172.21.15.43:0/4189304947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160101100 con 0x7fa160100670 2024-09-11T08:24:59.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.880+0000 7fa15f7fe640 1 --2- 172.21.15.43:0/4189304947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa160100a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa150009840 tx=0x7fa1500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c31129d42e89bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:24:59.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.880+0000 7fa15e7fc640 1 -- 172.21.15.43:0/4189304947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa15002add0 con 0x7fa160100670 2024-09-11T08:24:59.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.880+0000 7fa15e7fc640 1 -- 172.21.15.43:0/4189304947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fa150032890 con 0x7fa160100670 2024-09-11T08:24:59.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.880+0000 7fa15e7fc640 1 -- 172.21.15.43:0/4189304947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1500317a0 con 0x7fa160100670 2024-09-11T08:24:59.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.881+0000 7fa165fcc640 1 -- 172.21.15.43:0/4189304947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 msgr2=0x7fa160100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:24:59.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.881+0000 7fa165fcc640 1 --2- 172.21.15.43:0/4189304947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa160100a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa150009840 tx=0x7fa1500091d0 comp rx=0 tx=0).stop 2024-09-11T08:24:59.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.881+0000 7fa165fcc640 1 -- 172.21.15.43:0/4189304947 shutdown_connections 2024-09-11T08:24:59.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.881+0000 7fa165fcc640 1 --2- 172.21.15.43:0/4189304947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa160100a80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:24:59.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.881+0000 7fa165fcc640 1 -- 172.21.15.43:0/4189304947 >> 172.21.15.43:0/4189304947 conn(0x7fa1600fbe10 msgr2=0x7fa1600fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:24:59.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.881+0000 7fa165fcc640 1 -- 172.21.15.43:0/4189304947 shutdown_connections 2024-09-11T08:24:59.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.881+0000 7fa165fcc640 1 -- 172.21.15.43:0/4189304947 wait complete. 2024-09-11T08:24:59.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.881+0000 7fa165fcc640 1 Processor -- start 2024-09-11T08:24:59.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa165fcc640 1 -- start start 2024-09-11T08:24:59.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa165fcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa16010f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:24:59.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa165fcc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16010f7b0 con 0x7fa160100670 2024-09-11T08:24:59.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa15f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa16010f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:24:59.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa15f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa16010f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55982/0 (socket says 172.21.15.43:55982) 2024-09-11T08:24:59.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa15f7fe640 1 -- 172.21.15.43:0/2017610403 learned_addr learned my addr 172.21.15.43:0/2017610403 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:24:59.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa15f7fe640 1 -- 172.21.15.43:0/2017610403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa150009450 con 0x7fa160100670 2024-09-11T08:24:59.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa15f7fe640 1 --2- 172.21.15.43:0/2017610403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa16010f270 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa1500314e0 tx=0x7fa1500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:24:59.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.882+0000 7fa147fff640 1 -- 172.21.15.43:0/2017610403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa15003c400 con 0x7fa160100670 2024-09-11T08:24:59.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.883+0000 7fa147fff640 1 -- 172.21.15.43:0/2017610403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fa15003c9b0 con 0x7fa160100670 2024-09-11T08:24:59.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.883+0000 7fa147fff640 1 -- 172.21.15.43:0/2017610403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa15003a530 con 0x7fa160100670 2024-09-11T08:24:59.490 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.883+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16010f9b0 con 0x7fa160100670 2024-09-11T08:24:59.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.883+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16010fe20 con 0x7fa160100670 2024-09-11T08:24:59.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.883+0000 7fa147fff640 1 -- 172.21.15.43:0/2017610403 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa15003aa80 con 0x7fa160100670 2024-09-11T08:24:59.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.883+0000 7fa147fff640 1 -- 172.21.15.43:0/2017610403 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa150046c40 con 0x7fa160100670 2024-09-11T08:24:59.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.884+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16004b720 con 0x7fa160100670 2024-09-11T08:24:59.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.886+0000 7fa147fff640 1 -- 172.21.15.43:0/2017610403 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fa16010fe20 con 0x7fa160100670 2024-09-11T08:24:59.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.943+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fa160102350 con 0x7fa160100670 2024-09-11T08:24:59.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.945+0000 7fa147fff640 1 -- 172.21.15.43:0/2017610403 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa150037070 con 0x7fa160100670 2024-09-11T08:24:59.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.946+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 msgr2=0x7fa16010f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:24:59.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.946+0000 7fa165fcc640 1 --2- 172.21.15.43:0/2017610403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa16010f270 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa1500314e0 tx=0x7fa1500091d0 comp rx=0 tx=0).stop 2024-09-11T08:24:59.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.946+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 shutdown_connections 2024-09-11T08:24:59.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.946+0000 7fa165fcc640 1 --2- 172.21.15.43:0/2017610403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa160100670 0x7fa16010f270 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:24:59.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.946+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 >> 172.21.15.43:0/2017610403 conn(0x7fa1600fbe10 msgr2=0x7fa1600fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:24:59.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.946+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 shutdown_connections 2024-09-11T08:24:59.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:24:58.946+0000 7fa165fcc640 1 -- 172.21.15.43:0/2017610403 wait complete. 2024-09-11T08:24:59.493 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T08:24:59.493 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:24:59.494 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-09-11T08:24:59.494 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T08:24:59.494 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T08:24:59.662 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:24:59.662 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T08:24:59.820 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-327ee848-7017-11ef-bcea-c7b262605968.target.wants/ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-327ee848-7017-11ef-bcea-c7b262605968@.service. 2024-09-11T08:24:59.945 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:24:59 smithi043 systemd[1]: Starting Ceph mgr.y for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:25:00.285 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:24:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2017610403' entity='client.admin' 2024-09-11T08:25:00.286 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 podman[29594]: 2024-09-11 08:24:59.951632593 +0000 UTC m=+0.016160830 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:25:00.286 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 podman[29594]: 2024-09-11 08:25:00.066647078 +0000 UTC m=+0.131175314 container create 9155dd89112de829e89f87389a95f70463d32238e8d5495b6101adf7ce2ba5d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T08:25:00.286 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 podman[29594]: 2024-09-11 08:25:00.285817139 +0000 UTC m=+0.350345376 container init 9155dd89112de829e89f87389a95f70463d32238e8d5495b6101adf7ce2ba5d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T08:25:00.391 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:25:00.391 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-11T08:25:00.399 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:25:00.399 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T08:25:00.399 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-11T08:25:00.399 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:25:00.408 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:25:00.408 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-11T08:25:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:25:00.417 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T08:25:00.417 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-11T08:25:00.417 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T08:25:00.417 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-09-11T08:25:00.417 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-09-11T08:25:00.537 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 podman[29594]: 2024-09-11 08:25:00.290731573 +0000 UTC m=+0.355259816 container start 9155dd89112de829e89f87389a95f70463d32238e8d5495b6101adf7ce2ba5d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T08:25:00.537 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 bash[29594]: 9155dd89112de829e89f87389a95f70463d32238e8d5495b6101adf7ce2ba5d1 2024-09-11T08:25:00.537 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 systemd[1]: Started Ceph mgr.y for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:25:00.538 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:00.409+0000 7f1c77e91ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:25:00.538 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:00.469+0000 7f1c77e91ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:25:00.538 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:00.536+0000 7f1c77e91ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:25:00.808 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:00.665+0000 7f1c77e91ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:25:00.808 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:00.807+0000 7f1c77e91ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:25:01.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:01 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:01.079+0000 7f1c77e91ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:25:01.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:01 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:01.142+0000 7f1c77e91ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:25:01.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:01 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:01.310+0000 7f1c77e91ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:25:02.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1749829829' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:25:02.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:02 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:02.232+0000 7f1c77e91ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:25:02.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:02 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:02.302+0000 7f1c77e91ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:25:02.678 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:25:02.678 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:25:02.678 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "327ee848-7017-11ef-bcea-c7b262605968", 2024-09-11T08:25:02.678 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:25:02.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:25:02.680 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:25:02.681 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:25:02.682 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:24:43.167062+0000", 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:25:02.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:25:02.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.992+0000 7fb9b9654640 1 Processor -- start 2024-09-11T08:25:02.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.992+0000 7fb9b9654640 1 -- start start 2024-09-11T08:25:02.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.992+0000 7fb9b9654640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4103150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:02.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.992+0000 7fb9b9654640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4103690 con 0x7fb9b4075530 2024-09-11T08:25:02.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.993+0000 7fb9b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:02.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.993+0000 7fb9b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56010/0 (socket says 172.21.15.43:56010) 2024-09-11T08:25:02.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.993+0000 7fb9b2ffd640 1 -- 172.21.15.43:0/3267314108 learned_addr learned my addr 172.21.15.43:0/3267314108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:02.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.993+0000 7fb9b2ffd640 1 -- 172.21.15.43:0/3267314108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b41037d0 con 0x7fb9b4075530 2024-09-11T08:25:02.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.994+0000 7fb9b2ffd640 1 --2- 172.21.15.43:0/3267314108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4103150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009840 tx=0x7fb9a40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebca6740bb5b9e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:02.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.994+0000 7fb9b1ffb640 1 -- 172.21.15.43:0/3267314108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9a402add0 con 0x7fb9b4075530 2024-09-11T08:25:02.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.994+0000 7fb9b1ffb640 1 -- 172.21.15.43:0/3267314108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fb9a4032890 con 0x7fb9b4075530 2024-09-11T08:25:02.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.994+0000 7fb9b1ffb640 1 -- 172.21.15.43:0/3267314108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9a40317a0 con 0x7fb9b4075530 2024-09-11T08:25:02.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.995+0000 7fb9b9654640 1 -- 172.21.15.43:0/3267314108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 msgr2=0x7fb9b4103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:02.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.995+0000 7fb9b9654640 1 --2- 172.21.15.43:0/3267314108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4103150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009840 tx=0x7fb9a40091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:02.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.995+0000 7fb9b9654640 1 -- 172.21.15.43:0/3267314108 shutdown_connections 2024-09-11T08:25:02.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.995+0000 7fb9b9654640 1 --2- 172.21.15.43:0/3267314108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4103150 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:02.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.995+0000 7fb9b9654640 1 -- 172.21.15.43:0/3267314108 >> 172.21.15.43:0/3267314108 conn(0x7fb9b40fac60 msgr2=0x7fb9b40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:02.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.995+0000 7fb9b9654640 1 -- 172.21.15.43:0/3267314108 shutdown_connections 2024-09-11T08:25:02.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.995+0000 7fb9b9654640 1 -- 172.21.15.43:0/3267314108 wait complete. 2024-09-11T08:25:02.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.995+0000 7fb9b9654640 1 Processor -- start 2024-09-11T08:25:02.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b9654640 1 -- start start 2024-09-11T08:25:02.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b9654640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:02.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b9654640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4100fc0 con 0x7fb9b4075530 2024-09-11T08:25:02.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:02.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4100a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56018/0 (socket says 172.21.15.43:56018) 2024-09-11T08:25:02.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b2ffd640 1 -- 172.21.15.43:0/1749829829 learned_addr learned my addr 172.21.15.43:0/1749829829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:02.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b2ffd640 1 -- 172.21.15.43:0/1749829829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4009450 con 0x7fb9b4075530 2024-09-11T08:25:02.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b2ffd640 1 --2- 172.21.15.43:0/1749829829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4100a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40313e0 tx=0x7fb9a40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:02.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb99b7fe640 1 -- 172.21.15.43:0/1749829829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9a4032cd0 con 0x7fb9b4075530 2024-09-11T08:25:02.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb99b7fe640 1 -- 172.21.15.43:0/1749829829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fb9a403c850 con 0x7fb9b4075530 2024-09-11T08:25:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b40ff050 con 0x7fb9b4075530 2024-09-11T08:25:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.996+0000 7fb99b7fe640 1 -- 172.21.15.43:0/1749829829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9a403a3b0 con 0x7fb9b4075530 2024-09-11T08:25:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.997+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b40ff4c0 con 0x7fb9b4075530 2024-09-11T08:25:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.997+0000 7fb99b7fe640 1 -- 172.21.15.43:0/1749829829 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb9a403a900 con 0x7fb9b4075530 2024-09-11T08:25:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.997+0000 7fb99b7fe640 1 -- 172.21.15.43:0/1749829829 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb9a4046b60 con 0x7fb9b4075530 2024-09-11T08:25:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:01.998+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b405ed70 con 0x7fb9b4075530 2024-09-11T08:25:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.000+0000 7fb99b7fe640 1 -- 172.21.15.43:0/1749829829 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fb9a4037070 con 0x7fb9b4075530 2024-09-11T08:25:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.083+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb9b4104570 con 0x7fb9b4075530 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.083+0000 7fb99b7fe640 1 -- 172.21.15.43:0/1749829829 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7fb9a4046d70 con 0x7fb9b4075530 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.085+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 msgr2=0x7fb9b4100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.085+0000 7fb9b9654640 1 --2- 172.21.15.43:0/1749829829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4100a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40313e0 tx=0x7fb9a40091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.085+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 shutdown_connections 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.085+0000 7fb9b9654640 1 --2- 172.21.15.43:0/1749829829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b4075530 0x7fb9b4100a80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.085+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 >> 172.21.15.43:0/1749829829 conn(0x7fb9b40fac60 msgr2=0x7fb9b40fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.085+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 shutdown_connections 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:02.085+0000 7fb9b9654640 1 -- 172.21.15.43:0/1749829829 wait complete. 2024-09-11T08:25:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-09-11T08:25:03.361 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.057+0000 7f1c77e91ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:25:03.362 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.173+0000 7f1c77e91ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:25:03.362 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.298+0000 7f1c77e91ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:25:03.622 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.360+0000 7f1c77e91ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:25:03.622 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.491+0000 7f1c77e91ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:25:03.622 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.560+0000 7f1c77e91ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:25:03.622 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.620+0000 7f1c77e91ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:25:03.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.760+0000 7f1c77e91ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:25:03.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.826+0000 7f1c77e91ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:25:04.328 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.894+0000 7f1c77e91ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:25:04.329 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:03.978+0000 7f1c77e91ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:25:04.329 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:04.037+0000 7f1c77e91ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:25:04.329 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:04.100+0000 7f1c77e91ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:25:05.101 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:04.910+0000 7f1c77e91ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:25:05.101 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: Activating manager daemon y 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: mgrmap e2: y(active, starting, since 0.0031442s) 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: Manager daemon y is now available 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' 2024-09-11T08:25:05.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:04 smithi043 ceph-mon[29355]: from='mgr.14100 172.21.15.43:0/1253855170' entity='mgr.y' 2024-09-11T08:25:05.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:25:05.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:25:05.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "327ee848-7017-11ef-bcea-c7b262605968", 2024-09-11T08:25:05.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:25:05.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:25:05.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:25:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:25:05.861 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:05.861 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:25:05.861 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:05.861 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:25:05.861 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:25:05.861 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:25:05.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:25:05.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:25:05.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:25:05.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:24:43.167062+0000", 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.177+0000 7ff1c2a09640 1 Processor -- start 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.177+0000 7ff1c2a09640 1 -- start start 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.177+0000 7ff1c2a09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.177+0000 7ff1c2a09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc100f90 con 0x7ff1bc1005b0 2024-09-11T08:25:05.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.178+0000 7ff1bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.178+0000 7ff1bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc1009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59490/0 (socket says 172.21.15.43:59490) 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.178+0000 7ff1bbfff640 1 -- 172.21.15.43:0/2222038239 learned_addr learned my addr 172.21.15.43:0/2222038239 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.178+0000 7ff1bbfff640 1 -- 172.21.15.43:0/2222038239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc1017e0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.179+0000 7ff1bbfff640 1 --2- 172.21.15.43:0/2222038239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc1009c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009840 tx=0x7ff1ac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581c6d320419876e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.179+0000 7ff1baffd640 1 -- 172.21.15.43:0/2222038239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1ac02add0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.179+0000 7ff1baffd640 1 -- 172.21.15.43:0/2222038239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7ff1ac032890 con 0x7ff1bc1005b0 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.179+0000 7ff1baffd640 1 -- 172.21.15.43:0/2222038239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1ac0317a0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.179+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2222038239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 msgr2=0x7ff1bc1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.179+0000 7ff1c2a09640 1 --2- 172.21.15.43:0/2222038239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc1009c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009840 tx=0x7ff1ac0091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.180+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2222038239 shutdown_connections 2024-09-11T08:25:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.180+0000 7ff1c2a09640 1 --2- 172.21.15.43:0/2222038239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc1009c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.180+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2222038239 >> 172.21.15.43:0/2222038239 conn(0x7ff1bc0fbcb0 msgr2=0x7ff1bc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.180+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2222038239 shutdown_connections 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.180+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2222038239 wait complete. 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.180+0000 7ff1c2a09640 1 Processor -- start 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.180+0000 7ff1c2a09640 1 -- start start 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.181+0000 7ff1c2a09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc10f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.181+0000 7ff1c2a09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc10f7c0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.181+0000 7ff1bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc10f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.181+0000 7ff1bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc10f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59496/0 (socket says 172.21.15.43:59496) 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.181+0000 7ff1bbfff640 1 -- 172.21.15.43:0/2523721526 learned_addr learned my addr 172.21.15.43:0/2523721526 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.181+0000 7ff1bbfff640 1 -- 172.21.15.43:0/2523721526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac009450 con 0x7ff1bc1005b0 2024-09-11T08:25:05.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.181+0000 7ff1bbfff640 1 --2- 172.21.15.43:0/2523721526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc10f280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0313e0 tx=0x7ff1ac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.182+0000 7ff1b8ff9640 1 -- 172.21.15.43:0/2523721526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1ac032cd0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.182+0000 7ff1b8ff9640 1 -- 172.21.15.43:0/2523721526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7ff1ac03c850 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.182+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc10f9c0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.182+0000 7ff1b8ff9640 1 -- 172.21.15.43:0/2523721526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1ac03a3b0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.182+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc10fe30 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.183+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc05ed70 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.183+0000 7ff1b8ff9640 1 -- 172.21.15.43:0/2523721526 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 2) v1 ==== 45248+0+0 (secure 0 0 0) 0x7ff1ac03c400 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.183+0000 7ff1b8ff9640 1 -- 172.21.15.43:0/2523721526 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff1ac06c4f0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.187+0000 7ff1b8ff9640 1 -- 172.21.15.43:0/2523721526 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff1ac03c9c0 con 0x7ff1bc1005b0 2024-09-11T08:25:05.874 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.246+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff1bc102520 con 0x7ff1bc1005b0 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.246+0000 7ff1b8ff9640 1 -- 172.21.15.43:0/2523721526 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7ff1ac037030 con 0x7ff1bc1005b0 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.247+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 msgr2=0x7ff1bc10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.247+0000 7ff1c2a09640 1 --2- 172.21.15.43:0/2523721526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc10f280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0313e0 tx=0x7ff1ac0091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.247+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 shutdown_connections 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.247+0000 7ff1c2a09640 1 --2- 172.21.15.43:0/2523721526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1bc1005b0 0x7ff1bc10f280 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.247+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 >> 172.21.15.43:0/2523721526 conn(0x7ff1bc0fbcb0 msgr2=0x7ff1bc0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.248+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 shutdown_connections 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:05.248+0000 7ff1c2a09640 1 -- 172.21.15.43:0/2523721526 wait complete. 2024-09-11T08:25:05.875 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-09-11T08:25:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2523721526' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:25:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:05 smithi043 ceph-mon[29355]: mgrmap e3: y(active, since 1.00676s) 2024-09-11T08:25:08.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:07 smithi043 ceph-mon[29355]: mgrmap e4: y(active, since 2s) 2024-09-11T08:25:09.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:25:09.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:25:09.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "327ee848-7017-11ef-bcea-c7b262605968", 2024-09-11T08:25:09.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-11T08:25:09.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:25:09.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:25:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:25:09.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:24:43.167062+0000", 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:25:09.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.348+0000 7f84c0708640 1 Processor -- start 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.348+0000 7f84c0708640 1 -- start start 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.348+0000 7f84c0708640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b81009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.348+0000 7f84c0708640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8100f00 con 0x7f84b81005b0 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.348+0000 7f84be4a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b81009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.348+0000 7f84be4a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b81009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59502/0 (socket says 172.21.15.43:59502) 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.348+0000 7f84be4a4640 1 -- 172.21.15.43:0/1552773818 learned_addr learned my addr 172.21.15.43:0/1552773818 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.349+0000 7f84be4a4640 1 -- 172.21.15.43:0/1552773818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8101040 con 0x7f84b81005b0 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.349+0000 7f84be4a4640 1 --2- 172.21.15.43:0/1552773818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b81009c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009840 tx=0x7f84b40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7239000233c10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84bd4a2640 1 -- 172.21.15.43:0/1552773818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84b402add0 con 0x7f84b81005b0 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84bd4a2640 1 -- 172.21.15.43:0/1552773818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f84b4032890 con 0x7f84b81005b0 2024-09-11T08:25:09.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84bd4a2640 1 -- 172.21.15.43:0/1552773818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84b40317a0 con 0x7f84b81005b0 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84c0708640 1 -- 172.21.15.43:0/1552773818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 msgr2=0x7f84b81009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84c0708640 1 --2- 172.21.15.43:0/1552773818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b81009c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009840 tx=0x7f84b40091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84c0708640 1 -- 172.21.15.43:0/1552773818 shutdown_connections 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84c0708640 1 --2- 172.21.15.43:0/1552773818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b81009c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84c0708640 1 -- 172.21.15.43:0/1552773818 >> 172.21.15.43:0/1552773818 conn(0x7f84b80fbe10 msgr2=0x7f84b80fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84c0708640 1 -- 172.21.15.43:0/1552773818 shutdown_connections 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.350+0000 7f84c0708640 1 -- 172.21.15.43:0/1552773818 wait complete. 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.351+0000 7f84c0708640 1 Processor -- start 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.351+0000 7f84c0708640 1 -- start start 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.351+0000 7f84c0708640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b80774b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.351+0000 7f84c0708640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80779f0 con 0x7f84b81005b0 2024-09-11T08:25:09.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.351+0000 7f84be4a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b80774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.351+0000 7f84be4a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b80774b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59518/0 (socket says 172.21.15.43:59518) 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.351+0000 7f84be4a4640 1 -- 172.21.15.43:0/3676109867 learned_addr learned my addr 172.21.15.43:0/3676109867 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.352+0000 7f84be4a4640 1 -- 172.21.15.43:0/3676109867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4009450 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.352+0000 7f84be4a4640 1 --2- 172.21.15.43:0/3676109867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b80774b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f84b40314e0 tx=0x7f84b40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.352+0000 7f84aeffd640 1 -- 172.21.15.43:0/3676109867 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84b403c400 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.352+0000 7f84aeffd640 1 -- 172.21.15.43:0/3676109867 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f84b403c9b0 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.352+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b8075a80 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.352+0000 7f84aeffd640 1 -- 172.21.15.43:0/3676109867 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84b403a530 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.352+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b8075ef0 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.353+0000 7f84aeffd640 1 -- 172.21.15.43:0/3676109867 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 45446+0+0 (secure 0 0 0) 0x7f84b403c560 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.353+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b804b720 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.353+0000 7f84aeffd640 1 --2- 172.21.15.43:0/3676109867 >> v2:172.21.15.43:6800/3257998337 conn(0x7f84a4039430 0x7f84a403b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.353+0000 7f84aeffd640 1 -- 172.21.15.43:0/3676109867 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f84b406ca20 con 0x7f84b81005b0 2024-09-11T08:25:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.353+0000 7f84bdca3640 1 --2- 172.21.15.43:0/3676109867 >> v2:172.21.15.43:6800/3257998337 conn(0x7f84a4039430 0x7f84a403b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.354+0000 7f84bdca3640 1 --2- 172.21.15.43:0/3676109867 >> v2:172.21.15.43:6800/3257998337 conn(0x7f84a4039430 0x7f84a403b8e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84a8006fd0 tx=0x7f84a8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.358+0000 7f84aeffd640 1 -- 172.21.15.43:0/3676109867 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f84b403c810 con 0x7f84b81005b0 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.581+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f84b8076380 con 0x7f84b81005b0 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.581+0000 7f84aeffd640 1 -- 172.21.15.43:0/3676109867 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f84b4030b30 con 0x7f84b81005b0 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 >> v2:172.21.15.43:6800/3257998337 conn(0x7f84a4039430 msgr2=0x7f84a403b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 --2- 172.21.15.43:0/3676109867 >> v2:172.21.15.43:6800/3257998337 conn(0x7f84a4039430 0x7f84a403b8e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84a8006fd0 tx=0x7f84a8006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 msgr2=0x7f84b80774b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 --2- 172.21.15.43:0/3676109867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b80774b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f84b40314e0 tx=0x7f84b40091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 shutdown_connections 2024-09-11T08:25:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 --2- 172.21.15.43:0/3676109867 >> v2:172.21.15.43:6800/3257998337 conn(0x7f84a4039430 0x7f84a403b8e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:09.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 --2- 172.21.15.43:0/3676109867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b81005b0 0x7f84b80774b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:09.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 >> 172.21.15.43:0/3676109867 conn(0x7f84b80fbe10 msgr2=0x7f84b80fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:09.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 shutdown_connections 2024-09-11T08:25:09.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:08.583+0000 7f84c0708640 1 -- 172.21.15.43:0/3676109867 wait complete. 2024-09-11T08:25:09.141 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-09-11T08:25:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3676109867' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:25:10.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/626015072' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:25:10.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.524+0000 7f01ec614640 1 Processor -- start 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.524+0000 7f01ec614640 1 -- start start 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.524+0000 7f01ec614640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e40ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.524+0000 7f01ec614640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4100400 con 0x7f01e40ffab0 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.524+0000 7f01ea3b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e40ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.525+0000 7f01ea3b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e40ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59530/0 (socket says 172.21.15.43:59530) 2024-09-11T08:25:10.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.525+0000 7f01ea3b0640 1 -- 172.21.15.43:0/2487241870 learned_addr learned my addr 172.21.15.43:0/2487241870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.525+0000 7f01ea3b0640 1 -- 172.21.15.43:0/2487241870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e4100540 con 0x7f01e40ffab0 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.525+0000 7f01ea3b0640 1 --2- 172.21.15.43:0/2487241870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e40ffec0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009840 tx=0x7f01e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28d97fb31b6b79b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01e93ae640 1 -- 172.21.15.43:0/2487241870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01e002add0 con 0x7f01e40ffab0 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01e93ae640 1 -- 172.21.15.43:0/2487241870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f01e0032890 con 0x7f01e40ffab0 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01e93ae640 1 -- 172.21.15.43:0/2487241870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01e00317a0 con 0x7f01e40ffab0 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01ec614640 1 -- 172.21.15.43:0/2487241870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 msgr2=0x7f01e40ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01ec614640 1 --2- 172.21.15.43:0/2487241870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e40ffec0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009840 tx=0x7f01e00091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01ec614640 1 -- 172.21.15.43:0/2487241870 shutdown_connections 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01ec614640 1 --2- 172.21.15.43:0/2487241870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e40ffec0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01ec614640 1 -- 172.21.15.43:0/2487241870 >> 172.21.15.43:0/2487241870 conn(0x7f01e40fa440 msgr2=0x7f01e40fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01ec614640 1 -- 172.21.15.43:0/2487241870 shutdown_connections 2024-09-11T08:25:10.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.526+0000 7f01ec614640 1 -- 172.21.15.43:0/2487241870 wait complete. 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.527+0000 7f01ec614640 1 Processor -- start 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.527+0000 7f01ec614640 1 -- start start 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.527+0000 7f01ec614640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e410f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.527+0000 7f01ec614640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e410f7a0 con 0x7f01e40ffab0 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.527+0000 7f01ea3b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e410f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.527+0000 7f01ea3b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e410f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59534/0 (socket says 172.21.15.43:59534) 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.527+0000 7f01ea3b0640 1 -- 172.21.15.43:0/626015072 learned_addr learned my addr 172.21.15.43:0/626015072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.527+0000 7f01ea3b0640 1 -- 172.21.15.43:0/626015072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0009450 con 0x7f01e40ffab0 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.528+0000 7f01ea3b0640 1 --2- 172.21.15.43:0/626015072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e410f260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01e00314e0 tx=0x7f01e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.528+0000 7f01daffd640 1 -- 172.21.15.43:0/626015072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01e003c400 con 0x7f01e40ffab0 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.528+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e410f9a0 con 0x7f01e40ffab0 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.528+0000 7f01daffd640 1 -- 172.21.15.43:0/626015072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f01e003c9b0 con 0x7f01e40ffab0 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.528+0000 7f01daffd640 1 -- 172.21.15.43:0/626015072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01e003a530 con 0x7f01e40ffab0 2024-09-11T08:25:10.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.528+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e410fe10 con 0x7f01e40ffab0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.529+0000 7f01daffd640 1 -- 172.21.15.43:0/626015072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 45446+0+0 (secure 0 0 0) 0x7f01e003a7f0 con 0x7f01e40ffab0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.529+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e404b720 con 0x7f01e40ffab0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.529+0000 7f01daffd640 1 --2- 172.21.15.43:0/626015072 >> v2:172.21.15.43:6800/3257998337 conn(0x7f01d00393e0 0x7f01d003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.529+0000 7f01daffd640 1 -- 172.21.15.43:0/626015072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f01e006cb70 con 0x7f01e40ffab0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.529+0000 7f01e9baf640 1 --2- 172.21.15.43:0/626015072 >> v2:172.21.15.43:6800/3257998337 conn(0x7f01d00393e0 0x7f01d003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.529+0000 7f01e9baf640 1 --2- 172.21.15.43:0/626015072 >> v2:172.21.15.43:6800/3257998337 conn(0x7f01d00393e0 0x7f01d003b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01d4006fd0 tx=0x7f01d4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.534+0000 7f01daffd640 1 -- 172.21.15.43:0/626015072 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f01e0039d80 con 0x7f01e40ffab0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.698+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f01e4101280 con 0x7f01e40ffab0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.699+0000 7f01daffd640 1 -- 172.21.15.43:0/626015072 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+318 (secure 0 0 0) 0x7f01e0037020 con 0x7f01e40ffab0 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 >> v2:172.21.15.43:6800/3257998337 conn(0x7f01d00393e0 msgr2=0x7f01d003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 --2- 172.21.15.43:0/626015072 >> v2:172.21.15.43:6800/3257998337 conn(0x7f01d00393e0 0x7f01d003b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01d4006fd0 tx=0x7f01d4006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 msgr2=0x7f01e410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 --2- 172.21.15.43:0/626015072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e410f260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01e00314e0 tx=0x7f01e00091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:10.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 shutdown_connections 2024-09-11T08:25:10.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 --2- 172.21.15.43:0/626015072 >> v2:172.21.15.43:6800/3257998337 conn(0x7f01d00393e0 0x7f01d003b890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:10.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 --2- 172.21.15.43:0/626015072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40ffab0 0x7f01e410f260 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:10.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 >> 172.21.15.43:0/626015072 conn(0x7f01e40fa440 msgr2=0x7f01e40fb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:10.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 shutdown_connections 2024-09-11T08:25:10.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:09.701+0000 7f01ec614640 1 -- 172.21.15.43:0/626015072 wait complete. 2024-09-11T08:25:10.304 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-09-11T08:25:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2696600824' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T08:25:11.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ignoring --setuser ceph since I am not root 2024-09-11T08:25:11.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ignoring --setgroup ceph since I am not root 2024-09-11T08:25:11.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:11.108+0000 7efd5639bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:25:11.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:11.164+0000 7efd5639bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:25:11.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:11.232+0000 7efd5639bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:25:11.507 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:11.363+0000 7efd5639bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:25:11.507 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:11.506+0000 7efd5639bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:25:11.604 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.750+0000 7f25abbbf640 1 Processor -- start 2024-09-11T08:25:11.604 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.751+0000 7f25abbbf640 1 -- start start 2024-09-11T08:25:11.604 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.751+0000 7f25abbbf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a41031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:11.604 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.751+0000 7f25abbbf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41036f0 con 0x7f25a4074c20 2024-09-11T08:25:11.604 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.751+0000 7f25a995b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a41031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:11.604 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.751+0000 7f25a995b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a41031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59546/0 (socket says 172.21.15.43:59546) 2024-09-11T08:25:11.604 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.751+0000 7f25a995b640 1 -- 172.21.15.43:0/4016421271 learned_addr learned my addr 172.21.15.43:0/4016421271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:11.604 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.751+0000 7f25a995b640 1 -- 172.21.15.43:0/4016421271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a4103830 con 0x7f25a4074c20 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.752+0000 7f25a995b640 1 --2- 172.21.15.43:0/4016421271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a41031b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009840 tx=0x7f25a00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19be07b3af12ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.752+0000 7f25a8959640 1 -- 172.21.15.43:0/4016421271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25a002ae60 con 0x7f25a4074c20 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.752+0000 7f25a8959640 1 -- 172.21.15.43:0/4016421271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f25a0032890 con 0x7f25a4074c20 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.752+0000 7f25a8959640 1 -- 172.21.15.43:0/4016421271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25a00317a0 con 0x7f25a4074c20 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 -- 172.21.15.43:0/4016421271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 msgr2=0x7f25a41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 --2- 172.21.15.43:0/4016421271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a41031b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009840 tx=0x7f25a00091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 -- 172.21.15.43:0/4016421271 shutdown_connections 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 --2- 172.21.15.43:0/4016421271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a41031b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 -- 172.21.15.43:0/4016421271 >> 172.21.15.43:0/4016421271 conn(0x7f25a40fabc0 msgr2=0x7f25a40fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 -- 172.21.15.43:0/4016421271 shutdown_connections 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 -- 172.21.15.43:0/4016421271 wait complete. 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 Processor -- start 2024-09-11T08:25:11.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.753+0000 7f25abbbf640 1 -- start start 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25abbbf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a410f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25abbbf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a410f850 con 0x7f25a4074c20 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25a995b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a410f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25a995b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a410f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59552/0 (socket says 172.21.15.43:59552) 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25a995b640 1 -- 172.21.15.43:0/2696600824 learned_addr learned my addr 172.21.15.43:0/2696600824 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25a995b640 1 -- 172.21.15.43:0/2696600824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0009450 con 0x7f25a4074c20 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25a995b640 1 --2- 172.21.15.43:0/2696600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a410f310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25a00313e0 tx=0x7f25a00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f259a7fc640 1 -- 172.21.15.43:0/2696600824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25a0031be0 con 0x7f25a4074c20 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f259a7fc640 1 -- 172.21.15.43:0/2696600824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f25a0031d40 con 0x7f25a4074c20 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a410fa50 con 0x7f25a4074c20 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f259a7fc640 1 -- 172.21.15.43:0/2696600824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25a003bdc0 con 0x7f25a4074c20 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.754+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a410fec0 con 0x7f25a4074c20 2024-09-11T08:25:11.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.755+0000 7f259a7fc640 1 -- 172.21.15.43:0/2696600824 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 45446+0+0 (secure 0 0 0) 0x7f25a0039040 con 0x7f25a4074c20 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.755+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a404b720 con 0x7f25a4074c20 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.755+0000 7f259a7fc640 1 --2- 172.21.15.43:0/2696600824 >> v2:172.21.15.43:6800/3257998337 conn(0x7f25900393e0 0x7f259003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.755+0000 7f259a7fc640 1 -- 172.21.15.43:0/2696600824 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f25a006c180 con 0x7f25a4074c20 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.756+0000 7f25a915a640 1 --2- 172.21.15.43:0/2696600824 >> v2:172.21.15.43:6800/3257998337 conn(0x7f25900393e0 0x7f259003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.756+0000 7f25a915a640 1 --2- 172.21.15.43:0/2696600824 >> v2:172.21.15.43:6800/3257998337 conn(0x7f25900393e0 0x7f259003b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2594006fd0 tx=0x7f2594006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.760+0000 7f259a7fc640 1 -- 172.21.15.43:0/2696600824 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f25a003a9e0 con 0x7f25a4074c20 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.965+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f25a410fec0 con 0x7f25a4074c20 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.969+0000 7f259a7fc640 1 -- 172.21.15.43:0/2696600824 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 5) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f25a004ad40 con 0x7f25a4074c20 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.970+0000 7f259a7fc640 1 -- 172.21.15.43:0/2696600824 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f25a004cd70 con 0x7f25a4074c20 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 >> v2:172.21.15.43:6800/3257998337 conn(0x7f25900393e0 msgr2=0x7f259003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 --2- 172.21.15.43:0/2696600824 >> v2:172.21.15.43:6800/3257998337 conn(0x7f25900393e0 0x7f259003b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2594006fd0 tx=0x7f2594006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 msgr2=0x7f25a410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:11.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 --2- 172.21.15.43:0/2696600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a410f310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25a00313e0 tx=0x7f25a00091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:11.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 shutdown_connections 2024-09-11T08:25:11.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 --2- 172.21.15.43:0/2696600824 >> v2:172.21.15.43:6800/3257998337 conn(0x7f25900393e0 0x7f259003b890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:11.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 --2- 172.21.15.43:0/2696600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4074c20 0x7f25a410f310 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:11.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 >> 172.21.15.43:0/2696600824 conn(0x7f25a40fabc0 msgr2=0x7f25a40fb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:11.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 shutdown_connections 2024-09-11T08:25:11.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:10.973+0000 7f25abbbf640 1 -- 172.21.15.43:0/2696600824 wait complete. 2024-09-11T08:25:12.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2696600824' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T08:25:12.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-mon[29355]: mgrmap e5: y(active, since 6s) 2024-09-11T08:25:12.031 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:11.774+0000 7efd5639bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:25:12.031 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:11 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:11.838+0000 7efd5639bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:25:12.031 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:12 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:12.012+0000 7efd5639bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.029+0000 7f95c1e87640 1 Processor -- start 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.029+0000 7f95c1e87640 1 -- start start 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.029+0000 7f95c1e87640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc105530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.029+0000 7f95c1e87640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc100f20 con 0x7f95bc074ca0 2024-09-11T08:25:12.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.030+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc105530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.030+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc105530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59570/0 (socket says 172.21.15.43:59570) 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.030+0000 7f95bb7fe640 1 -- 172.21.15.43:0/3062323549 learned_addr learned my addr 172.21.15.43:0/3062323549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.030+0000 7f95bb7fe640 1 -- 172.21.15.43:0/3062323549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc105a70 con 0x7f95bc074ca0 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.031+0000 7f95bb7fe640 1 --2- 172.21.15.43:0/3062323549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc105530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009840 tx=0x7f95ac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0df91b8e846702 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.031+0000 7f95ba7fc640 1 -- 172.21.15.43:0/3062323549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95ac02add0 con 0x7f95bc074ca0 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.031+0000 7f95ba7fc640 1 -- 172.21.15.43:0/3062323549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f95ac032890 con 0x7f95bc074ca0 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.031+0000 7f95ba7fc640 1 -- 172.21.15.43:0/3062323549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95ac0317a0 con 0x7f95bc074ca0 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 -- 172.21.15.43:0/3062323549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 msgr2=0x7f95bc105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 --2- 172.21.15.43:0/3062323549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc105530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009840 tx=0x7f95ac0091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 -- 172.21.15.43:0/3062323549 shutdown_connections 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 --2- 172.21.15.43:0/3062323549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc105530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 -- 172.21.15.43:0/3062323549 >> 172.21.15.43:0/3062323549 conn(0x7f95bc0fabc0 msgr2=0x7f95bc0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 -- 172.21.15.43:0/3062323549 shutdown_connections 2024-09-11T08:25:12.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 -- 172.21.15.43:0/3062323549 wait complete. 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 Processor -- start 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.032+0000 7f95c1e87640 1 -- start start 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95c1e87640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc1114b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95c1e87640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc1119f0 con 0x7f95bc074ca0 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc1114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc1114b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59574/0 (socket says 172.21.15.43:59574) 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95bb7fe640 1 -- 172.21.15.43:0/3293357227 learned_addr learned my addr 172.21.15.43:0/3293357227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95bb7fe640 1 -- 172.21.15.43:0/3293357227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac009450 con 0x7f95bc074ca0 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95bb7fe640 1 --2- 172.21.15.43:0/3293357227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc1114b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0313e0 tx=0x7f95ac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95a3fff640 1 -- 172.21.15.43:0/3293357227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95ac032cd0 con 0x7f95bc074ca0 2024-09-11T08:25:12.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95a3fff640 1 -- 172.21.15.43:0/3293357227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f95ac03c850 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95bc111bf0 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95a3fff640 1 -- 172.21.15.43:0/3293357227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95ac03a3b0 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.033+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc112060 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.034+0000 7f95a3fff640 1 -- 172.21.15.43:0/3293357227 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f95ac03a670 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.034+0000 7f95a3fff640 1 --2- 172.21.15.43:0/3293357227 >> v2:172.21.15.43:6800/3257998337 conn(0x7f95a4039430 0x7f95a403b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.034+0000 7f95a3fff640 1 -- 172.21.15.43:0/3293357227 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f95ac06c910 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.034+0000 7f95baffd640 1 -- 172.21.15.43:0/3293357227 >> v2:172.21.15.43:6800/3257998337 conn(0x7f95a4039430 msgr2=0x7f95a403b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3257998337 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.035+0000 7f95baffd640 1 --2- 172.21.15.43:0/3293357227 >> v2:172.21.15.43:6800/3257998337 conn(0x7f95a4039430 0x7f95a403b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.035+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc05ed70 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.039+0000 7f95a3fff640 1 -- 172.21.15.43:0/3293357227 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f95ac037030 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.236+0000 7f95baffd640 1 -- 172.21.15.43:0/3293357227 >> v2:172.21.15.43:6800/3257998337 conn(0x7f95a4039430 msgr2=0x7f95a403b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3257998337 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.236+0000 7f95baffd640 1 --2- 172.21.15.43:0/3293357227 >> v2:172.21.15.43:6800/3257998337 conn(0x7f95a4039430 0x7f95a403b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.337+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f95bc04b720 con 0x7f95bc074ca0 2024-09-11T08:25:12.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.338+0000 7f95a3fff640 1 -- 172.21.15.43:0/3293357227 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f95ac039af0 con 0x7f95bc074ca0 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 >> v2:172.21.15.43:6800/3257998337 conn(0x7f95a4039430 msgr2=0x7f95a403b8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 --2- 172.21.15.43:0/3293357227 >> v2:172.21.15.43:6800/3257998337 conn(0x7f95a4039430 0x7f95a403b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 msgr2=0x7f95bc1114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 --2- 172.21.15.43:0/3293357227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc1114b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0313e0 tx=0x7f95ac0091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 shutdown_connections 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 --2- 172.21.15.43:0/3293357227 >> v2:172.21.15.43:6800/3257998337 conn(0x7f95a4039430 0x7f95a403b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 --2- 172.21.15.43:0/3293357227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95bc074ca0 0x7f95bc1114b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 >> 172.21.15.43:0/3293357227 conn(0x7f95bc0fabc0 msgr2=0x7f95bc0fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 shutdown_connections 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:12.340+0000 7f95c1e87640 1 -- 172.21.15.43:0/3293357227 wait complete. 2024-09-11T08:25:12.868 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-11T08:25:12.869 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-09-11T08:25:13.287 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3293357227' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:25:13.288 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:12 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:12.975+0000 7efd5639bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:25:13.288 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:13.044+0000 7efd5639bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:25:14.094 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:13.790+0000 7efd5639bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:25:14.094 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:13.906+0000 7efd5639bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:25:14.094 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.031+0000 7efd5639bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:25:14.354 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.093+0000 7efd5639bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:25:14.355 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.224+0000 7efd5639bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:25:14.355 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.293+0000 7efd5639bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:25:14.355 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.353+0000 7efd5639bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:25:14.628 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.493+0000 7efd5639bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:25:14.628 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.559+0000 7efd5639bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:25:14.628 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.627+0000 7efd5639bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:25:14.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.711+0000 7efd5639bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:25:14.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.771+0000 7efd5639bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:25:14.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:14 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:14.833+0000 7efd5639bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:25:15.687 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:15.633+0000 7efd5639bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:25:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: Active manager daemon y restarted 2024-09-11T08:25:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: Activating manager daemon y 2024-09-11T08:25:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T08:25:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: mgrmap e6: y(active, starting, since 0.00308336s) 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: Manager daemon y is now available 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:25:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:15 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:25:16.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:16 smithi043 ceph-mon[29355]: Found migration_current of "None". Setting to last migration. 2024-09-11T08:25:16.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:16 smithi043 ceph-mon[29355]: mgrmap e7: y(active, since 1.00574s) 2024-09-11T08:25:17.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:25:17.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T08:25:17.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:25:17.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.286+0000 7fdd3efe8640 1 Processor -- start 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.286+0000 7fdd3efe8640 1 -- start start 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.286+0000 7fdd3efe8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.286+0000 7fdd3efe8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38100fc0 con 0x7fdd38074c00 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.286+0000 7fdd3cd84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.287+0000 7fdd3cd84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36334/0 (socket says 172.21.15.43:36334) 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.287+0000 7fdd3cd84640 1 -- 172.21.15.43:0/1880971595 learned_addr learned my addr 172.21.15.43:0/1880971595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.287+0000 7fdd3cd84640 1 -- 172.21.15.43:0/1880971595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38105b10 con 0x7fdd38074c00 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.288+0000 7fdd3cd84640 1 --2- 172.21.15.43:0/1880971595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381055d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009840 tx=0x7fdd2c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a09468985c885ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.288+0000 7fdd377fe640 1 -- 172.21.15.43:0/1880971595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd2c02add0 con 0x7fdd38074c00 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.288+0000 7fdd377fe640 1 -- 172.21.15.43:0/1880971595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fdd2c032890 con 0x7fdd38074c00 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.288+0000 7fdd377fe640 1 -- 172.21.15.43:0/1880971595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd2c0317a0 con 0x7fdd38074c00 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.288+0000 7fdd3efe8640 1 -- 172.21.15.43:0/1880971595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 msgr2=0x7fdd381055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:17.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.288+0000 7fdd3efe8640 1 --2- 172.21.15.43:0/1880971595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381055d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009840 tx=0x7fdd2c0091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 -- 172.21.15.43:0/1880971595 shutdown_connections 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 --2- 172.21.15.43:0/1880971595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381055d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 -- 172.21.15.43:0/1880971595 >> 172.21.15.43:0/1880971595 conn(0x7fdd380fabc0 msgr2=0x7fdd380fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 -- 172.21.15.43:0/1880971595 shutdown_connections 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 -- 172.21.15.43:0/1880971595 wait complete. 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 Processor -- start 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 -- start start 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381114f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.289+0000 7fdd3efe8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38111a30 con 0x7fdd38074c00 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd3cd84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd3cd84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381114f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36342/0 (socket says 172.21.15.43:36342) 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd3cd84640 1 -- 172.21.15.43:0/695946254 learned_addr learned my addr 172.21.15.43:0/695946254 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:17.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd3cd84640 1 -- 172.21.15.43:0/695946254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c009450 con 0x7fdd38074c00 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd3cd84640 1 --2- 172.21.15.43:0/695946254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381114f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0313e0 tx=0x7fdd2c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd2c031e70 con 0x7fdd38074c00 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fdd2c03c850 con 0x7fdd38074c00 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd38111c30 con 0x7fdd38074c00 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd2c03a580 con 0x7fdd38074c00 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.290+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd381120a0 con 0x7fdd38074c00 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.291+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 45457+0+0 (secure 0 0 0) 0x7fdd2c03a800 con 0x7fdd38074c00 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.291+0000 7fdd357fa640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 0x7fdd2803b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.291+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 --> v2:172.21.15.43:6800/3257998337 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd2803bfb0 con 0x7fdd28039430 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.292+0000 7fdd37fff640 1 -- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 msgr2=0x7fdd2803b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3257998337 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.292+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdd2c06c9d0 con 0x7fdd38074c00 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.292+0000 7fdd37fff640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 0x7fdd2803b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:25:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.493+0000 7fdd37fff640 1 -- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 msgr2=0x7fdd2803b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3257998337 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.493+0000 7fdd37fff640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 0x7fdd2803b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.894+0000 7fdd37fff640 1 -- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 msgr2=0x7fdd2803b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3257998337 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:13.894+0000 7fdd37fff640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 0x7fdd2803b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:14.695+0000 7fdd37fff640 1 -- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 msgr2=0x7fdd2803b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3257998337 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:14.695+0000 7fdd37fff640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 0x7fdd2803b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:15.636+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 6) v1 ==== 45259+0+0 (secure 0 0 0) 0x7fdd2c02fd80 con 0x7fdd38074c00 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:15.636+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 msgr2=0x7fdd2803b8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:15.636+0000 7fdd357fa640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 0x7fdd2803b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.638+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 7) v1 ==== 45351+0+0 (secure 0 0 0) 0x7fdd2c039310 con 0x7fdd38074c00 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.638+0000 7fdd357fa640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/225101565 conn(0x7fdd2803cdc0 0x7fdd2803f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.638+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 --> v2:172.21.15.43:6800/225101565 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd2803bfb0 con 0x7fdd2803cdc0 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.638+0000 7fdd37fff640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/225101565 conn(0x7fdd2803cdc0 0x7fdd2803f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.639+0000 7fdd37fff640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/225101565 conn(0x7fdd2803cdc0 0x7fdd2803f1a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdd24003940 tx=0x7fdd24009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:17.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.640+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mgr.14118 v2:172.21.15.43:6800/225101565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fdd2803bfb0 con 0x7fdd2803cdc0 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.645+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 --> v2:172.21.15.43:6800/225101565 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdd381120a0 con 0x7fdd2803cdc0 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.645+0000 7fdd357fa640 1 -- 172.21.15.43:0/695946254 <== mgr.14118 v2:172.21.15.43:6800/225101565 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fdd381120a0 con 0x7fdd2803cdc0 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.645+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/225101565 conn(0x7fdd2803cdc0 msgr2=0x7fdd2803f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.645+0000 7fdd3efe8640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/225101565 conn(0x7fdd2803cdc0 0x7fdd2803f1a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdd24003940 tx=0x7fdd24009350 comp rx=0 tx=0).stop 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.645+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 msgr2=0x7fdd381114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.645+0000 7fdd3efe8640 1 --2- 172.21.15.43:0/695946254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381114f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0313e0 tx=0x7fdd2c0091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.646+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 shutdown_connections 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.646+0000 7fdd3efe8640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/225101565 conn(0x7fdd2803cdc0 0x7fdd2803f1a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.646+0000 7fdd3efe8640 1 --2- 172.21.15.43:0/695946254 >> v2:172.21.15.43:6800/3257998337 conn(0x7fdd28039430 0x7fdd2803b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.646+0000 7fdd3efe8640 1 --2- 172.21.15.43:0/695946254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38074c00 0x7fdd381114f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.646+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 >> 172.21.15.43:0/695946254 conn(0x7fdd380fabc0 msgr2=0x7fdd380fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.646+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 shutdown_connections 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:16.646+0000 7fdd3efe8640 1 -- 172.21.15.43:0/695946254 wait complete. 2024-09-11T08:25:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-09-11T08:25:17.120 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-09-11T08:25:17.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:17 smithi043 ceph-mon[29355]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T08:25:17.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:17 smithi043 ceph-mon[29355]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:18.087+0000 7efd08749640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:25:18] ENGINE Error in HTTPServer.serve 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: Traceback (most recent call last): 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._connections.run(self.expiration_interval) 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._run(expiration_interval) 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:25:18.294 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: s = self.context.wrap_socket( 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: return self.sslsocket_class._create( 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self.do_handshake() 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._sslobj.do_handshake() 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:25:18.295 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:18.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.535+0000 7fe28839c640 1 Processor -- start 2024-09-11T08:25:18.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.535+0000 7fe28839c640 1 -- start start 2024-09-11T08:25:18.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.536+0000 7fe28839c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280103130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:18.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.536+0000 7fe28839c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe280103670 con 0x7fe280074ca0 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.536+0000 7fe286138640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.536+0000 7fe286138640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36396/0 (socket says 172.21.15.43:36396) 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.536+0000 7fe286138640 1 -- 172.21.15.43:0/616089362 learned_addr learned my addr 172.21.15.43:0/616089362 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.536+0000 7fe286138640 1 -- 172.21.15.43:0/616089362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2801037b0 con 0x7fe280074ca0 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.537+0000 7fe286138640 1 --2- 172.21.15.43:0/616089362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280103130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009840 tx=0x7fe27c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f8a87d301b6ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.537+0000 7fe285136640 1 -- 172.21.15.43:0/616089362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe27c02add0 con 0x7fe280074ca0 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.537+0000 7fe285136640 1 -- 172.21.15.43:0/616089362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fe27c032890 con 0x7fe280074ca0 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.537+0000 7fe285136640 1 -- 172.21.15.43:0/616089362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe27c0317a0 con 0x7fe280074ca0 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 -- 172.21.15.43:0/616089362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 msgr2=0x7fe280103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:18.301 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 --2- 172.21.15.43:0/616089362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280103130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009840 tx=0x7fe27c0091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 -- 172.21.15.43:0/616089362 shutdown_connections 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 --2- 172.21.15.43:0/616089362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280103130 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 -- 172.21.15.43:0/616089362 >> 172.21.15.43:0/616089362 conn(0x7fe2800fabc0 msgr2=0x7fe2800fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 -- 172.21.15.43:0/616089362 shutdown_connections 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 -- 172.21.15.43:0/616089362 wait complete. 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 Processor -- start 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.538+0000 7fe28839c640 1 -- start start 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe28839c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe28839c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe280100fa0 con 0x7fe280074ca0 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe286138640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe286138640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36408/0 (socket says 172.21.15.43:36408) 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe286138640 1 -- 172.21.15.43:0/1209643002 learned_addr learned my addr 172.21.15.43:0/1209643002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe286138640 1 -- 172.21.15.43:0/1209643002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c009450 con 0x7fe280074ca0 2024-09-11T08:25:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe286138640 1 --2- 172.21.15.43:0/1209643002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280100a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0313e0 tx=0x7fe27c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe276ffd640 1 -- 172.21.15.43:0/1209643002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe27c031e70 con 0x7fe280074ca0 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe276ffd640 1 -- 172.21.15.43:0/1209643002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fe27c03c850 con 0x7fe280074ca0 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2800ff030 con 0x7fe280074ca0 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe276ffd640 1 -- 172.21.15.43:0/1209643002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe27c03a580 con 0x7fe280074ca0 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.539+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2800ff4a0 con 0x7fe280074ca0 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.540+0000 7fe276ffd640 1 -- 172.21.15.43:0/1209643002 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 7) v1 ==== 45351+0+0 (secure 0 0 0) 0x7fe27c03cc50 con 0x7fe280074ca0 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.540+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28004b720 con 0x7fe280074ca0 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.541+0000 7fe276ffd640 1 --2- 172.21.15.43:0/1209643002 >> v2:172.21.15.43:6800/225101565 conn(0x7fe26c039360 0x7fe26c03b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.541+0000 7fe276ffd640 1 -- 172.21.15.43:0/1209643002 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fe27c06c6b0 con 0x7fe280074ca0 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.541+0000 7fe285937640 1 --2- 172.21.15.43:0/1209643002 >> v2:172.21.15.43:6800/225101565 conn(0x7fe26c039360 0x7fe26c03b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.541+0000 7fe285937640 1 --2- 172.21.15.43:0/1209643002 >> v2:172.21.15.43:6800/225101565 conn(0x7fe26c039360 0x7fe26c03b810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe270006fd0 tx=0x7fe270006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.545+0000 7fe276ffd640 1 -- 172.21.15.43:0/1209643002 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe27c046460 con 0x7fe280074ca0 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.691+0000 7fe276ffd640 1 -- 172.21.15.43:0/1209643002 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7fe27c03c400 con 0x7fe280074ca0 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.722+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 --> v2:172.21.15.43:6800/225101565 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2800ff9a0 con 0x7fe26c039360 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.732+0000 7fe276ffd640 1 -- 172.21.15.43:0/1209643002 <== mgr.14118 v2:172.21.15.43:6800/225101565 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe2800ff9a0 con 0x7fe26c039360 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 >> v2:172.21.15.43:6800/225101565 conn(0x7fe26c039360 msgr2=0x7fe26c03b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 --2- 172.21.15.43:0/1209643002 >> v2:172.21.15.43:6800/225101565 conn(0x7fe26c039360 0x7fe26c03b810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe270006fd0 tx=0x7fe270006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 msgr2=0x7fe280100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 --2- 172.21.15.43:0/1209643002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280100a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0313e0 tx=0x7fe27c0091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 shutdown_connections 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 --2- 172.21.15.43:0/1209643002 >> v2:172.21.15.43:6800/225101565 conn(0x7fe26c039360 0x7fe26c03b810 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:18.304 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 --2- 172.21.15.43:0/1209643002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe280074ca0 0x7fe280100a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:18.305 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 >> 172.21.15.43:0/1209643002 conn(0x7fe2800fabc0 msgr2=0x7fe2800fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:18.305 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 shutdown_connections 2024-09-11T08:25:18.305 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:17.734+0000 7fe28839c640 1 -- 172.21.15.43:0/1209643002 wait complete. 2024-09-11T08:25:18.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-mon[29355]: mgrmap e8: y(active, since 2s) 2024-09-11T08:25:18.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-mon[29355]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:18.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-mon[29355]: [11/Sep/2024:08:25:17] ENGINE Bus STARTING 2024-09-11T08:25:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:18 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.749+0000 7f8e98b9d640 1 Processor -- start 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.749+0000 7f8e98b9d640 1 -- start start 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.749+0000 7f8e98b9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e941033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.749+0000 7f8e98b9d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94075630 con 0x7f8e94100fc0 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.749+0000 7f8e937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e941033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.749+0000 7f8e937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e941033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36424/0 (socket says 172.21.15.43:36424) 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.749+0000 7f8e937fe640 1 -- 172.21.15.43:0/1668892974 learned_addr learned my addr 172.21.15.43:0/1668892974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.750+0000 7f8e937fe640 1 -- 172.21.15.43:0/1668892974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e94075770 con 0x7f8e94100fc0 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.750+0000 7f8e937fe640 1 --2- 172.21.15.43:0/1668892974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e941033a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009e50 tx=0x7f8e84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25536ce0fad545a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e927fc640 1 -- 172.21.15.43:0/1668892974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e84031400 con 0x7f8e94100fc0 2024-09-11T08:25:19.491 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e927fc640 1 -- 172.21.15.43:0/1668892974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f8e8402d050 con 0x7f8e94100fc0 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e927fc640 1 -- 172.21.15.43:0/1668892974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e840076e0 con 0x7f8e94100fc0 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1668892974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 msgr2=0x7f8e941033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e98b9d640 1 --2- 172.21.15.43:0/1668892974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e941033a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009e50 tx=0x7f8e84009840 comp rx=0 tx=0).stop 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1668892974 shutdown_connections 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e98b9d640 1 --2- 172.21.15.43:0/1668892974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e941033a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1668892974 >> 172.21.15.43:0/1668892974 conn(0x7f8e940fab20 msgr2=0x7f8e940fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1668892974 shutdown_connections 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.751+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1668892974 wait complete. 2024-09-11T08:25:19.492 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.752+0000 7f8e98b9d640 1 Processor -- start 2024-09-11T08:25:19.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.752+0000 7f8e98b9d640 1 -- start start 2024-09-11T08:25:19.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.752+0000 7f8e98b9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e94111250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:19.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.752+0000 7f8e98b9d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94111790 con 0x7f8e94100fc0 2024-09-11T08:25:19.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.752+0000 7f8e937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e94111250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:19.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.752+0000 7f8e937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e94111250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36428/0 (socket says 172.21.15.43:36428) 2024-09-11T08:25:19.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.752+0000 7f8e937fe640 1 -- 172.21.15.43:0/1610232001 learned_addr learned my addr 172.21.15.43:0/1610232001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:19.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.752+0000 7f8e937fe640 1 -- 172.21.15.43:0/1610232001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e84009ae0 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.753+0000 7f8e937fe640 1 --2- 172.21.15.43:0/1610232001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e94111250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e8402abe0 tx=0x7f8e84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.753+0000 7f8e8bfff640 1 -- 172.21.15.43:0/1610232001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e840077f0 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.753+0000 7f8e8bfff640 1 -- 172.21.15.43:0/1610232001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f8e84007da0 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.753+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e94111990 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.753+0000 7f8e8bfff640 1 -- 172.21.15.43:0/1610232001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e84045ca0 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.753+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e94111e00 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.754+0000 7f8e8bfff640 1 -- 172.21.15.43:0/1610232001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f8e84038040 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.754+0000 7f8e8bfff640 1 --2- 172.21.15.43:0/1610232001 >> v2:172.21.15.43:6800/225101565 conn(0x7f8e80039530 0x7f8e8003b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.754+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9405ed70 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.754+0000 7f8e8bfff640 1 -- 172.21.15.43:0/1610232001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f8e84040030 con 0x7f8e94100fc0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.754+0000 7f8e92ffd640 1 --2- 172.21.15.43:0/1610232001 >> v2:172.21.15.43:6800/225101565 conn(0x7f8e80039530 0x7f8e8003b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.754+0000 7f8e92ffd640 1 --2- 172.21.15.43:0/1610232001 >> v2:172.21.15.43:6800/225101565 conn(0x7f8e80039530 0x7f8e8003b9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c006fd0 tx=0x7f8e7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:19.494 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.759+0000 7f8e8bfff640 1 -- 172.21.15.43:0/1610232001 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8e840382f0 con 0x7f8e94100fc0 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.925+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 --> v2:172.21.15.43:6800/225101565 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e94111e00 con 0x7f8e80039530 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.926+0000 7f8e8bfff640 1 -- 172.21.15.43:0/1610232001 <== mgr.14118 v2:172.21.15.43:6800/225101565 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f8e94111e00 con 0x7f8e80039530 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 >> v2:172.21.15.43:6800/225101565 conn(0x7f8e80039530 msgr2=0x7f8e8003b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 --2- 172.21.15.43:0/1610232001 >> v2:172.21.15.43:6800/225101565 conn(0x7f8e80039530 0x7f8e8003b9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c006fd0 tx=0x7f8e7c006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 msgr2=0x7f8e94111250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 --2- 172.21.15.43:0/1610232001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e94111250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e8402abe0 tx=0x7f8e84009840 comp rx=0 tx=0).stop 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 shutdown_connections 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 --2- 172.21.15.43:0/1610232001 >> v2:172.21.15.43:6800/225101565 conn(0x7f8e80039530 0x7f8e8003b9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 --2- 172.21.15.43:0/1610232001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e94100fc0 0x7f8e94111250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 >> 172.21.15.43:0/1610232001 conn(0x7f8e940fab20 msgr2=0x7f8e940fb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 shutdown_connections 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:18.928+0000 7f8e98b9d640 1 -- 172.21.15.43:0/1610232001 wait complete. 2024-09-11T08:25:19.495 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:19 smithi043 ceph-mon[29355]: [11/Sep/2024:08:25:18] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:19 smithi043 ceph-mon[29355]: [11/Sep/2024:08:25:18] ENGINE Bus STARTED 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:19 smithi043 ceph-mon[29355]: [11/Sep/2024:08:25:18] ENGINE Error in HTTPServer.serve 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:25:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:25:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:19 smithi043 ceph-mon[29355]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:20.631 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: Generating public/private rsa key pair. 2024-09-11T08:25:20.631 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: Your identification has been saved in /tmp/tmpcv_3vyuk/key 2024-09-11T08:25:20.631 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: Your public key has been saved in /tmp/tmpcv_3vyuk/key.pub 2024-09-11T08:25:20.631 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: The key fingerprint is: 2024-09-11T08:25:20.631 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: SHA256:5RuD2l/d1dLbGBGaQZzkIn7/BX2mHJn6evNFhrEz86c ceph-327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: The key's randomart image is: 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: +---[RSA 3072]----+ 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | ++.. | 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | .o+ . | 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | . o + o | 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | . = . X.| 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | S = & X| 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | o . = + ^=| 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | . . . + =.B| 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | . . oooo| 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: | . .oEo.| 2024-09-11T08:25:20.632 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: +----[SHA256]-----+ 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.890+0000 7ff37805a640 1 Processor -- start 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.890+0000 7ff37805a640 1 -- start start 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.890+0000 7ff37805a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff3701004c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.890+0000 7ff37805a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff370100a90 con 0x7ff3701000b0 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.890+0000 7ff375df6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff3701004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.890+0000 7ff375df6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff3701004c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36436/0 (socket says 172.21.15.43:36436) 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.890+0000 7ff375df6640 1 -- 172.21.15.43:0/993300150 learned_addr learned my addr 172.21.15.43:0/993300150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.891+0000 7ff375df6640 1 -- 172.21.15.43:0/993300150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff370100bd0 con 0x7ff3701000b0 2024-09-11T08:25:20.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.891+0000 7ff375df6640 1 --2- 172.21.15.43:0/993300150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff3701004c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00a9f0 tx=0x7ff36c00a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda77992e5a32367 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff374df4640 1 -- 172.21.15.43:0/993300150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff36c030510 con 0x7ff3701000b0 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff374df4640 1 -- 172.21.15.43:0/993300150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7ff36c02e050 con 0x7ff3701000b0 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff374df4640 1 -- 172.21.15.43:0/993300150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff36c0328e0 con 0x7ff3701000b0 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff37805a640 1 -- 172.21.15.43:0/993300150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 msgr2=0x7ff3701004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff37805a640 1 --2- 172.21.15.43:0/993300150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff3701004c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00a9f0 tx=0x7ff36c00a360 comp rx=0 tx=0).stop 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff37805a640 1 -- 172.21.15.43:0/993300150 shutdown_connections 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff37805a640 1 --2- 172.21.15.43:0/993300150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff3701004c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff37805a640 1 -- 172.21.15.43:0/993300150 >> 172.21.15.43:0/993300150 conn(0x7ff3700f9fa0 msgr2=0x7ff3700fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff37805a640 1 -- 172.21.15.43:0/993300150 shutdown_connections 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.892+0000 7ff37805a640 1 -- 172.21.15.43:0/993300150 wait complete. 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.893+0000 7ff37805a640 1 Processor -- start 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.893+0000 7ff37805a640 1 -- start start 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.893+0000 7ff37805a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff37010f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.893+0000 7ff37805a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37010f7d0 con 0x7ff3701000b0 2024-09-11T08:25:20.915 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff375df6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff37010f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff375df6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff37010f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36444/0 (socket says 172.21.15.43:36444) 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff375df6640 1 -- 172.21.15.43:0/164662467 learned_addr learned my addr 172.21.15.43:0/164662467 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff375df6640 1 -- 172.21.15.43:0/164662467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c00a600 con 0x7ff3701000b0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff375df6640 1 --2- 172.21.15.43:0/164662467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff37010f290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff36c032620 tx=0x7ff36c00a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff3667fc640 1 -- 172.21.15.43:0/164662467 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff36c03b430 con 0x7ff3701000b0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff3667fc640 1 -- 172.21.15.43:0/164662467 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7ff36c03b9e0 con 0x7ff3701000b0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37010f9d0 con 0x7ff3701000b0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff3667fc640 1 -- 172.21.15.43:0/164662467 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff36c03a4d0 con 0x7ff3701000b0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.894+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37010fe40 con 0x7ff3701000b0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.895+0000 7ff3667fc640 1 -- 172.21.15.43:0/164662467 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7ff36c03a630 con 0x7ff3701000b0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.895+0000 7ff3667fc640 1 --2- 172.21.15.43:0/164662467 >> v2:172.21.15.43:6800/225101565 conn(0x7ff35c039470 0x7ff35c03b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.895+0000 7ff3667fc640 1 -- 172.21.15.43:0/164662467 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7ff36c06dbb0 con 0x7ff3701000b0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.896+0000 7ff3755f5640 1 --2- 172.21.15.43:0/164662467 >> v2:172.21.15.43:6800/225101565 conn(0x7ff35c039470 0x7ff35c03b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:20.916 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.896+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37005ed70 con 0x7ff3701000b0 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.896+0000 7ff3755f5640 1 --2- 172.21.15.43:0/164662467 >> v2:172.21.15.43:6800/225101565 conn(0x7ff35c039470 0x7ff35c03b920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff360006fd0 tx=0x7ff360006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:19.900+0000 7ff3667fc640 1 -- 172.21.15.43:0/164662467 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff36c039ad0 con 0x7ff3701000b0 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.065+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 --> v2:172.21.15.43:6800/225101565 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3700008d0 con 0x7ff35c039470 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.326+0000 7ff3667fc640 1 -- 172.21.15.43:0/164662467 <== mgr.14118 v2:172.21.15.43:6800/225101565 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3700008d0 con 0x7ff35c039470 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 >> v2:172.21.15.43:6800/225101565 conn(0x7ff35c039470 msgr2=0x7ff35c03b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 --2- 172.21.15.43:0/164662467 >> v2:172.21.15.43:6800/225101565 conn(0x7ff35c039470 0x7ff35c03b920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff360006fd0 tx=0x7ff360006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 msgr2=0x7ff37010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 --2- 172.21.15.43:0/164662467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff37010f290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff36c032620 tx=0x7ff36c00a360 comp rx=0 tx=0).stop 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 shutdown_connections 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 --2- 172.21.15.43:0/164662467 >> v2:172.21.15.43:6800/225101565 conn(0x7ff35c039470 0x7ff35c03b920 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 --2- 172.21.15.43:0/164662467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3701000b0 0x7ff37010f290 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:20.917 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 >> 172.21.15.43:0/164662467 conn(0x7ff3700f9fa0 msgr2=0x7ff370103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:20.918 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 shutdown_connections 2024-09-11T08:25:20.918 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:20.328+0000 7ff37805a640 1 -- 172.21.15.43:0/164662467 wait complete. 2024-09-11T08:25:21.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:21 smithi043 ceph-mon[29355]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:21.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:21 smithi043 ceph-mon[29355]: Generating ssh key... 2024-09-11T08:25:21.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:21 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:21.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:21 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC52znb6C5Dyya2KXz/qcQiTztSIK7rrdWaIDAqiKqNukZ4xG6UPMGKq6TPON4YGQJNr9D2/T3ukTsLyZvDtU5FQ851DILdgDFC+hD6E/R/cGQqAt/WNiHd5VIDACu7jQJjnOtQAqYSCkcPGlI8nCh9MW/Nc465XweufMRRvZnmwvYbSYiu/3bfX9Czof5RnGmlINSE9rdqNwy9Jt5YwKdfMFYGtToEYzRBguLaVGThw4sGnOKeNCRZQ8wmwqmuS6pnaMcrveOb9wgB8HbwX8RoadGaRtFogU0YV9l7aUKb57ZjJwzoJCZIyrQ8CR+eyIJXmHTWD48AqFFl8MYYa6GDZHp7UzOvrWOdH9ld9HWLm0aXgtMeD853LgQwcm5SAZTAKLNBk5xHm4ALKnkXp/MXOaaqBv+K+W3CN4WnF0DtkoYXmdwfK73/UpiMyelBZO4WFIHgoHXLqq/Fy/GOvzA6OYnHgLYqSLGBLwncP5Z5H7ETQmBX0kLu7yMWXOIMCts= ceph-327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.387+0000 7ff64ec36640 1 Processor -- start 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.387+0000 7ff64ec36640 1 -- start start 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.388+0000 7ff64ec36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff648100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.388+0000 7ff64ec36640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648101010 con 0x7ff6481006c0 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.388+0000 7ff64c9d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff648100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.388+0000 7ff64c9d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff648100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36454/0 (socket says 172.21.15.43:36454) 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.388+0000 7ff64c9d2640 1 -- 172.21.15.43:0/101046634 learned_addr learned my addr 172.21.15.43:0/101046634 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.388+0000 7ff64c9d2640 1 -- 172.21.15.43:0/101046634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648101150 con 0x7ff6481006c0 2024-09-11T08:25:22.121 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.389+0000 7ff64c9d2640 1 --2- 172.21.15.43:0/101046634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff648100ad0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff640009840 tx=0x7ff6400091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b87d0d3d526135 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.389+0000 7ff6477fe640 1 -- 172.21.15.43:0/101046634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff64002ae60 con 0x7ff6481006c0 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.389+0000 7ff6477fe640 1 -- 172.21.15.43:0/101046634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7ff640032890 con 0x7ff6481006c0 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.389+0000 7ff6477fe640 1 -- 172.21.15.43:0/101046634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6400317a0 con 0x7ff6481006c0 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.389+0000 7ff64ec36640 1 -- 172.21.15.43:0/101046634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 msgr2=0x7ff648100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.389+0000 7ff64ec36640 1 --2- 172.21.15.43:0/101046634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff648100ad0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff640009840 tx=0x7ff6400091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.390+0000 7ff64ec36640 1 -- 172.21.15.43:0/101046634 shutdown_connections 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.390+0000 7ff64ec36640 1 --2- 172.21.15.43:0/101046634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff648100ad0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.390+0000 7ff64ec36640 1 -- 172.21.15.43:0/101046634 >> 172.21.15.43:0/101046634 conn(0x7ff6480fbea0 msgr2=0x7ff6480fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.390+0000 7ff64ec36640 1 -- 172.21.15.43:0/101046634 shutdown_connections 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.390+0000 7ff64ec36640 1 -- 172.21.15.43:0/101046634 wait complete. 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.390+0000 7ff64ec36640 1 Processor -- start 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.390+0000 7ff64ec36640 1 -- start start 2024-09-11T08:25:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64ec36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff64810f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64ec36640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64810f7d0 con 0x7ff6481006c0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64c9d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff64810f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64c9d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff64810f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36460/0 (socket says 172.21.15.43:36460) 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64c9d2640 1 -- 172.21.15.43:0/1470356881 learned_addr learned my addr 172.21.15.43:0/1470356881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64c9d2640 1 -- 172.21.15.43:0/1470356881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff640009450 con 0x7ff6481006c0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64c9d2640 1 --2- 172.21.15.43:0/1470356881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff64810f290 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6400313e0 tx=0x7ff6400091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff6457fa640 1 -- 172.21.15.43:0/1470356881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff640031c20 con 0x7ff6481006c0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff6457fa640 1 -- 172.21.15.43:0/1470356881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7ff640031d80 con 0x7ff6481006c0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64810f9d0 con 0x7ff6481006c0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff6457fa640 1 -- 172.21.15.43:0/1470356881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff64003a390 con 0x7ff6481006c0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.391+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64810fe40 con 0x7ff6481006c0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.392+0000 7ff6457fa640 1 -- 172.21.15.43:0/1470356881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7ff640039040 con 0x7ff6481006c0 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.392+0000 7ff6457fa640 1 --2- 172.21.15.43:0/1470356881 >> v2:172.21.15.43:6800/225101565 conn(0x7ff634039430 0x7ff63403b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:22.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.392+0000 7ff6457fa640 1 -- 172.21.15.43:0/1470356881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7ff64006d180 con 0x7ff6481006c0 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.393+0000 7ff647fff640 1 --2- 172.21.15.43:0/1470356881 >> v2:172.21.15.43:6800/225101565 conn(0x7ff634039430 0x7ff63403b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.393+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64805ed70 con 0x7ff6481006c0 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.393+0000 7ff647fff640 1 --2- 172.21.15.43:0/1470356881 >> v2:172.21.15.43:6800/225101565 conn(0x7ff634039430 0x7ff63403b8e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff63800ad90 tx=0x7ff638009410 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.397+0000 7ff6457fa640 1 -- 172.21.15.43:0/1470356881 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff64003f080 con 0x7ff6481006c0 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.566+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 --> v2:172.21.15.43:6800/225101565 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6481024a0 con 0x7ff634039430 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.567+0000 7ff6457fa640 1 -- 172.21.15.43:0/1470356881 <== mgr.14118 v2:172.21.15.43:6800/225101565 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7ff6481024a0 con 0x7ff634039430 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 >> v2:172.21.15.43:6800/225101565 conn(0x7ff634039430 msgr2=0x7ff63403b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 --2- 172.21.15.43:0/1470356881 >> v2:172.21.15.43:6800/225101565 conn(0x7ff634039430 0x7ff63403b8e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff63800ad90 tx=0x7ff638009410 comp rx=0 tx=0).stop 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 msgr2=0x7ff64810f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 --2- 172.21.15.43:0/1470356881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff64810f290 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6400313e0 tx=0x7ff6400091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 shutdown_connections 2024-09-11T08:25:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 --2- 172.21.15.43:0/1470356881 >> v2:172.21.15.43:6800/225101565 conn(0x7ff634039430 0x7ff63403b8e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:22.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 --2- 172.21.15.43:0/1470356881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6481006c0 0x7ff64810f290 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:22.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 >> 172.21.15.43:0/1470356881 conn(0x7ff6480fbea0 msgr2=0x7ff6480fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:22.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 shutdown_connections 2024-09-11T08:25:22.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:21.569+0000 7ff64ec36640 1 -- 172.21.15.43:0/1470356881 wait complete. 2024-09-11T08:25:22.125 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:25:22.125 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T08:25:22.127 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-09-11T08:25:22.382 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:22 smithi043 ceph-mon[29355]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:23 smithi043 ceph-mon[29355]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:24.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:24 smithi043 ceph-mon[29355]: Deploying cephadm binary to smithi043 2024-09-11T08:25:26.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-09-11T08:25:26.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.618+0000 7f9d49e4c640 1 Processor -- start 2024-09-11T08:25:26.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.618+0000 7f9d49e4c640 1 -- start start 2024-09-11T08:25:26.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.618+0000 7f9d49e4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d441009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:26.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.618+0000 7f9d49e4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d44100f00 con 0x7f9d441005b0 2024-09-11T08:25:26.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.618+0000 7f9d437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d441009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:26.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.618+0000 7f9d437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d441009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46354/0 (socket says 172.21.15.43:46354) 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.618+0000 7f9d437fe640 1 -- 172.21.15.43:0/3139907697 learned_addr learned my addr 172.21.15.43:0/3139907697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.619+0000 7f9d437fe640 1 -- 172.21.15.43:0/3139907697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d44101040 con 0x7f9d441005b0 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.619+0000 7f9d437fe640 1 --2- 172.21.15.43:0/3139907697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d441009c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009840 tx=0x7f9d340091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9920c40287c858d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d427fc640 1 -- 172.21.15.43:0/3139907697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d3402add0 con 0x7f9d441005b0 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d427fc640 1 -- 172.21.15.43:0/3139907697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f9d34032890 con 0x7f9d441005b0 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d427fc640 1 -- 172.21.15.43:0/3139907697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d340317a0 con 0x7f9d441005b0 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3139907697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 msgr2=0x7f9d441009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d49e4c640 1 --2- 172.21.15.43:0/3139907697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d441009c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009840 tx=0x7f9d340091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3139907697 shutdown_connections 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d49e4c640 1 --2- 172.21.15.43:0/3139907697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d441009c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3139907697 >> 172.21.15.43:0/3139907697 conn(0x7f9d440fbe10 msgr2=0x7f9d440fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3139907697 shutdown_connections 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.620+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3139907697 wait complete. 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.621+0000 7f9d49e4c640 1 Processor -- start 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.621+0000 7f9d49e4c640 1 -- start start 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.621+0000 7f9d49e4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d4410f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.621+0000 7f9d49e4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4410f780 con 0x7f9d441005b0 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.621+0000 7f9d437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d4410f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.621+0000 7f9d437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d4410f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46362/0 (socket says 172.21.15.43:46362) 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.621+0000 7f9d437fe640 1 -- 172.21.15.43:0/3742400914 learned_addr learned my addr 172.21.15.43:0/3742400914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.622+0000 7f9d437fe640 1 -- 172.21.15.43:0/3742400914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d34009450 con 0x7f9d441005b0 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.622+0000 7f9d437fe640 1 --2- 172.21.15.43:0/3742400914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d4410f240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9d340313e0 tx=0x7f9d340091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.622+0000 7f9d3bfff640 1 -- 172.21.15.43:0/3742400914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d34031e70 con 0x7f9d441005b0 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.622+0000 7f9d3bfff640 1 -- 172.21.15.43:0/3742400914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f9d3403c850 con 0x7f9d441005b0 2024-09-11T08:25:26.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.622+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4410f980 con 0x7f9d441005b0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.622+0000 7f9d3bfff640 1 -- 172.21.15.43:0/3742400914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d3403a540 con 0x7f9d441005b0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.622+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4410fdf0 con 0x7f9d441005b0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.623+0000 7f9d3bfff640 1 -- 172.21.15.43:0/3742400914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f9d3403a7c0 con 0x7f9d441005b0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.623+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4404b720 con 0x7f9d441005b0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.623+0000 7f9d3bfff640 1 --2- 172.21.15.43:0/3742400914 >> v2:172.21.15.43:6800/225101565 conn(0x7f9d30039430 0x7f9d3003b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.623+0000 7f9d3bfff640 1 -- 172.21.15.43:0/3742400914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f9d3406c690 con 0x7f9d441005b0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.623+0000 7f9d42ffd640 1 --2- 172.21.15.43:0/3742400914 >> v2:172.21.15.43:6800/225101565 conn(0x7f9d30039430 0x7f9d3003b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.624+0000 7f9d42ffd640 1 --2- 172.21.15.43:0/3742400914 >> v2:172.21.15.43:6800/225101565 conn(0x7f9d30039430 0x7f9d3003b8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c006fd0 tx=0x7f9d2c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.628+0000 7f9d3bfff640 1 -- 172.21.15.43:0/3742400914 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f9d34039d80 con 0x7f9d441005b0 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:22.800+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 --> v2:172.21.15.43:6800/225101565 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d4410fdf0 con 0x7f9d30039430 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.207+0000 7f9d3bfff640 1 -- 172.21.15.43:0/3742400914 <== mgr.14118 v2:172.21.15.43:6800/225101565 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9d4410fdf0 con 0x7f9d30039430 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 >> v2:172.21.15.43:6800/225101565 conn(0x7f9d30039430 msgr2=0x7f9d3003b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 --2- 172.21.15.43:0/3742400914 >> v2:172.21.15.43:6800/225101565 conn(0x7f9d30039430 0x7f9d3003b8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c006fd0 tx=0x7f9d2c006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 msgr2=0x7f9d4410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 --2- 172.21.15.43:0/3742400914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d4410f240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9d340313e0 tx=0x7f9d340091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 shutdown_connections 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 --2- 172.21.15.43:0/3742400914 >> v2:172.21.15.43:6800/225101565 conn(0x7f9d30039430 0x7f9d3003b8e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 --2- 172.21.15.43:0/3742400914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d441005b0 0x7f9d4410f240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 >> 172.21.15.43:0/3742400914 conn(0x7f9d440fbe10 msgr2=0x7f9d440fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 shutdown_connections 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:26.210+0000 7f9d49e4c640 1 -- 172.21.15.43:0/3742400914 wait complete. 2024-09-11T08:25:26.792 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-09-11T08:25:27.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:27 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:27.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:27 smithi043 ceph-mon[29355]: Added host smithi043 2024-09-11T08:25:27.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:27 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:28.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.221+0000 7fac34a1c640 1 Processor -- start 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.221+0000 7fac34a1c640 1 -- start start 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.221+0000 7fac34a1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac301009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.221+0000 7fac34a1c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac30100f00 con 0x7fac301005b0 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.221+0000 7fac2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac301009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.221+0000 7fac2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac301009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46374/0 (socket says 172.21.15.43:46374) 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.221+0000 7fac2e59c640 1 -- 172.21.15.43:0/2536200703 learned_addr learned my addr 172.21.15.43:0/2536200703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.222+0000 7fac2e59c640 1 -- 172.21.15.43:0/2536200703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac30101040 con 0x7fac301005b0 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.222+0000 7fac2e59c640 1 --2- 172.21.15.43:0/2536200703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac301009c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fac20009840 tx=0x7fac200091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700350f6936545e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac2d59a640 1 -- 172.21.15.43:0/2536200703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac2002add0 con 0x7fac301005b0 2024-09-11T08:25:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac2d59a640 1 -- 172.21.15.43:0/2536200703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fac20032890 con 0x7fac301005b0 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac2d59a640 1 -- 172.21.15.43:0/2536200703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac200317a0 con 0x7fac301005b0 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac34a1c640 1 -- 172.21.15.43:0/2536200703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 msgr2=0x7fac301009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac34a1c640 1 --2- 172.21.15.43:0/2536200703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac301009c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fac20009840 tx=0x7fac200091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac34a1c640 1 -- 172.21.15.43:0/2536200703 shutdown_connections 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac34a1c640 1 --2- 172.21.15.43:0/2536200703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac301009c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac34a1c640 1 -- 172.21.15.43:0/2536200703 >> 172.21.15.43:0/2536200703 conn(0x7fac300fbe10 msgr2=0x7fac300fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac34a1c640 1 -- 172.21.15.43:0/2536200703 shutdown_connections 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.223+0000 7fac34a1c640 1 -- 172.21.15.43:0/2536200703 wait complete. 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.224+0000 7fac34a1c640 1 Processor -- start 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.224+0000 7fac34a1c640 1 -- start start 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.224+0000 7fac34a1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac300774b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:28.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.224+0000 7fac34a1c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300779f0 con 0x7fac301005b0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.224+0000 7fac2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac300774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.224+0000 7fac2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac300774b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46390/0 (socket says 172.21.15.43:46390) 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.224+0000 7fac2e59c640 1 -- 172.21.15.43:0/2163600396 learned_addr learned my addr 172.21.15.43:0/2163600396 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.224+0000 7fac2e59c640 1 -- 172.21.15.43:0/2163600396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20009450 con 0x7fac301005b0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.225+0000 7fac2e59c640 1 --2- 172.21.15.43:0/2163600396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac300774b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fac200314e0 tx=0x7fac200091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.225+0000 7fac1effd640 1 -- 172.21.15.43:0/2163600396 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac2003c400 con 0x7fac301005b0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.225+0000 7fac1effd640 1 -- 172.21.15.43:0/2163600396 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fac2003c9b0 con 0x7fac301005b0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.225+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac30075a80 con 0x7fac301005b0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.225+0000 7fac1effd640 1 -- 172.21.15.43:0/2163600396 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac2003a530 con 0x7fac301005b0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.225+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac30075ef0 con 0x7fac301005b0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.226+0000 7fac1effd640 1 -- 172.21.15.43:0/2163600396 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7fac2003a7a0 con 0x7fac301005b0 2024-09-11T08:25:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.226+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3005ed70 con 0x7fac301005b0 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.226+0000 7fac1effd640 1 --2- 172.21.15.43:0/2163600396 >> v2:172.21.15.43:6800/225101565 conn(0x7fac18039470 0x7fac1803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.226+0000 7fac1effd640 1 -- 172.21.15.43:0/2163600396 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fac2006c9e0 con 0x7fac301005b0 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.226+0000 7fac2dd9b640 1 --2- 172.21.15.43:0/2163600396 >> v2:172.21.15.43:6800/225101565 conn(0x7fac18039470 0x7fac1803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.226+0000 7fac2dd9b640 1 --2- 172.21.15.43:0/2163600396 >> v2:172.21.15.43:6800/225101565 conn(0x7fac18039470 0x7fac1803b920 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fac24006fd0 tx=0x7fac24006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.230+0000 7fac1effd640 1 -- 172.21.15.43:0/2163600396 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fac20046460 con 0x7fac301005b0 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.409+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 --> v2:172.21.15.43:6800/225101565 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fac30076360 con 0x7fac18039470 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.413+0000 7fac1effd640 1 -- 172.21.15.43:0/2163600396 <== mgr.14118 v2:172.21.15.43:6800/225101565 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fac30076360 con 0x7fac18039470 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 >> v2:172.21.15.43:6800/225101565 conn(0x7fac18039470 msgr2=0x7fac1803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:28.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 --2- 172.21.15.43:0/2163600396 >> v2:172.21.15.43:6800/225101565 conn(0x7fac18039470 0x7fac1803b920 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fac24006fd0 tx=0x7fac24006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 msgr2=0x7fac300774b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 --2- 172.21.15.43:0/2163600396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac300774b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fac200314e0 tx=0x7fac200091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 shutdown_connections 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 --2- 172.21.15.43:0/2163600396 >> v2:172.21.15.43:6800/225101565 conn(0x7fac18039470 0x7fac1803b920 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 --2- 172.21.15.43:0/2163600396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac301005b0 0x7fac300774b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 >> 172.21.15.43:0/2163600396 conn(0x7fac300fbe10 msgr2=0x7fac300fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 shutdown_connections 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:27.415+0000 7fac34a1c640 1 -- 172.21.15.43:0/2163600396 wait complete. 2024-09-11T08:25:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-09-11T08:25:28.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:28 smithi043 ceph-mon[29355]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:28.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:28 smithi043 ceph-mon[29355]: Saving service mon spec with placement count:5 2024-09-11T08:25:28.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:28 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:29.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T08:25:29.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.713+0000 7f3d715e8640 1 Processor -- start 2024-09-11T08:25:29.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.713+0000 7f3d715e8640 1 -- start start 2024-09-11T08:25:29.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.714+0000 7f3d715e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c0ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:29.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.714+0000 7f3d715e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0ffdf0 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.714+0000 7f3d6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c0ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.714+0000 7f3d6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c0ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46402/0 (socket says 172.21.15.43:46402) 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.714+0000 7f3d6affd640 1 -- 172.21.15.43:0/1377011682 learned_addr learned my addr 172.21.15.43:0/1377011682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.714+0000 7f3d6affd640 1 -- 172.21.15.43:0/1377011682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c0fff30 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.715+0000 7f3d6affd640 1 --2- 172.21.15.43:0/1377011682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c0ff8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009840 tx=0x7f3d640091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ada09da3612ba64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.715+0000 7f3d69ffb640 1 -- 172.21.15.43:0/1377011682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6402add0 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.715+0000 7f3d69ffb640 1 -- 172.21.15.43:0/1377011682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f3d64032890 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.715+0000 7f3d69ffb640 1 -- 172.21.15.43:0/1377011682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d640317a0 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.715+0000 7f3d715e8640 1 -- 172.21.15.43:0/1377011682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 msgr2=0x7f3d6c0ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.715+0000 7f3d715e8640 1 --2- 172.21.15.43:0/1377011682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c0ff8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009840 tx=0x7f3d640091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 -- 172.21.15.43:0/1377011682 shutdown_connections 2024-09-11T08:25:29.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 --2- 172.21.15.43:0/1377011682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c0ff8b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 -- 172.21.15.43:0/1377011682 >> 172.21.15.43:0/1377011682 conn(0x7f3d6c0faca0 msgr2=0x7f3d6c0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 -- 172.21.15.43:0/1377011682 shutdown_connections 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 -- 172.21.15.43:0/1377011682 wait complete. 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 Processor -- start 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 -- start start 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c105680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.716+0000 7f3d715e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c105bc0 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c105680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c105680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46416/0 (socket says 172.21.15.43:46416) 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d6affd640 1 -- 172.21.15.43:0/2957339839 learned_addr learned my addr 172.21.15.43:0/2957339839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d6affd640 1 -- 172.21.15.43:0/2957339839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64009450 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d6affd640 1 --2- 172.21.15.43:0/2957339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c105680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d640313e0 tx=0x7f3d640091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d637fe640 1 -- 172.21.15.43:0/2957339839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d64031e70 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c103c50 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d637fe640 1 -- 172.21.15.43:0/2957339839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f3d6403c850 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d637fe640 1 -- 172.21.15.43:0/2957339839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6403a540 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.717+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c1040c0 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.718+0000 7f3d637fe640 1 -- 172.21.15.43:0/2957339839 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f3d6403a7c0 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.718+0000 7f3d637fe640 1 --2- 172.21.15.43:0/2957339839 >> v2:172.21.15.43:6800/225101565 conn(0x7f3d58039430 0x7f3d5803b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.718+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c04b720 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.719+0000 7f3d637fe640 1 -- 172.21.15.43:0/2957339839 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f3d6406c800 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.719+0000 7f3d6a7fc640 1 --2- 172.21.15.43:0/2957339839 >> v2:172.21.15.43:6800/225101565 conn(0x7f3d58039430 0x7f3d5803b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.719+0000 7f3d6a7fc640 1 --2- 172.21.15.43:0/2957339839 >> v2:172.21.15.43:6800/225101565 conn(0x7f3d58039430 0x7f3d5803b8e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d54006fd0 tx=0x7f3d54006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.724+0000 7f3d637fe640 1 -- 172.21.15.43:0/2957339839 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3d64037030 con 0x7f3d6c0ff4a0 2024-09-11T08:25:29.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.901+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 --> v2:172.21.15.43:6800/225101565 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3d6c104590 con 0x7f3d58039430 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.905+0000 7f3d637fe640 1 -- 172.21.15.43:0/2957339839 <== mgr.14118 v2:172.21.15.43:6800/225101565 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3d6c104590 con 0x7f3d58039430 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 >> v2:172.21.15.43:6800/225101565 conn(0x7f3d58039430 msgr2=0x7f3d5803b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 --2- 172.21.15.43:0/2957339839 >> v2:172.21.15.43:6800/225101565 conn(0x7f3d58039430 0x7f3d5803b8e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d54006fd0 tx=0x7f3d54006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 msgr2=0x7f3d6c105680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 --2- 172.21.15.43:0/2957339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c105680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d640313e0 tx=0x7f3d640091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 shutdown_connections 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 --2- 172.21.15.43:0/2957339839 >> v2:172.21.15.43:6800/225101565 conn(0x7f3d58039430 0x7f3d5803b8e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 --2- 172.21.15.43:0/2957339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d6c0ff4a0 0x7f3d6c105680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 >> 172.21.15.43:0/2957339839 conn(0x7f3d6c0faca0 msgr2=0x7f3d6c0fb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 shutdown_connections 2024-09-11T08:25:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:28.907+0000 7f3d715e8640 1 -- 172.21.15.43:0/2957339839 wait complete. 2024-09-11T08:25:29.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:29 smithi043 ceph-mon[29355]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:29.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:29 smithi043 ceph-mon[29355]: Saving service mgr spec with placement count:2 2024-09-11T08:25:29.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:29 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:31.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.570+0000 7f75e7078640 1 Processor -- start 2024-09-11T08:25:31.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.570+0000 7f75e7078640 1 -- start start 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.571+0000 7f75e7078640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e0105590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.571+0000 7f75e7078640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0100f80 con 0x7f75e0074c40 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.571+0000 7f75e4e14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e0105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.571+0000 7f75e4e14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e0105590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46420/0 (socket says 172.21.15.43:46420) 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.571+0000 7f75e4e14640 1 -- 172.21.15.43:0/833262383 learned_addr learned my addr 172.21.15.43:0/833262383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.571+0000 7f75e4e14640 1 -- 172.21.15.43:0/833262383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0105ad0 con 0x7f75e0074c40 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.572+0000 7f75e4e14640 1 --2- 172.21.15.43:0/833262383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e0105590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009840 tx=0x7f75d40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95a6ceb79e35d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.572+0000 7f75df7fe640 1 -- 172.21.15.43:0/833262383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d402add0 con 0x7f75e0074c40 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.572+0000 7f75df7fe640 1 -- 172.21.15.43:0/833262383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f75d40328b0 con 0x7f75e0074c40 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.572+0000 7f75df7fe640 1 -- 172.21.15.43:0/833262383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d4031690 con 0x7f75e0074c40 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.573+0000 7f75e7078640 1 -- 172.21.15.43:0/833262383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 msgr2=0x7f75e0105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.573+0000 7f75e7078640 1 --2- 172.21.15.43:0/833262383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e0105590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009840 tx=0x7f75d40091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.573+0000 7f75e7078640 1 -- 172.21.15.43:0/833262383 shutdown_connections 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.573+0000 7f75e7078640 1 --2- 172.21.15.43:0/833262383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e0105590 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.573+0000 7f75e7078640 1 -- 172.21.15.43:0/833262383 >> 172.21.15.43:0/833262383 conn(0x7f75e00fabc0 msgr2=0x7f75e00fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.573+0000 7f75e7078640 1 -- 172.21.15.43:0/833262383 shutdown_connections 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.573+0000 7f75e7078640 1 -- 172.21.15.43:0/833262383 wait complete. 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.574+0000 7f75e7078640 1 Processor -- start 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.574+0000 7f75e7078640 1 -- start start 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.574+0000 7f75e7078640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e01114d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.574+0000 7f75e7078640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0111a10 con 0x7f75e0074c40 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.574+0000 7f75e4e14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e01114d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.574+0000 7f75e4e14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e01114d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46422/0 (socket says 172.21.15.43:46422) 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.574+0000 7f75e4e14640 1 -- 172.21.15.43:0/1195819817 learned_addr learned my addr 172.21.15.43:0/1195819817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:31.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.574+0000 7f75e4e14640 1 -- 172.21.15.43:0/1195819817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4009450 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.575+0000 7f75e4e14640 1 --2- 172.21.15.43:0/1195819817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e01114d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75d4000c00 tx=0x7f75d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.575+0000 7f75dd7fa640 1 -- 172.21.15.43:0/1195819817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d4031c50 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.575+0000 7f75dd7fa640 1 -- 172.21.15.43:0/1195819817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f75d4031db0 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.575+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e0111c10 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.575+0000 7f75dd7fa640 1 -- 172.21.15.43:0/1195819817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d403a3a0 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.575+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e0112080 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.576+0000 7f75dd7fa640 1 -- 172.21.15.43:0/1195819817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f75d4039070 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.576+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e004b720 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.576+0000 7f75dd7fa640 1 --2- 172.21.15.43:0/1195819817 >> v2:172.21.15.43:6800/225101565 conn(0x7f75d0039470 0x7f75d003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.576+0000 7f75dd7fa640 1 -- 172.21.15.43:0/1195819817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f75d406c8e0 con 0x7f75e0074c40 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.576+0000 7f75dffff640 1 --2- 172.21.15.43:0/1195819817 >> v2:172.21.15.43:6800/225101565 conn(0x7f75d0039470 0x7f75d003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.577+0000 7f75dffff640 1 --2- 172.21.15.43:0/1195819817 >> v2:172.21.15.43:6800/225101565 conn(0x7f75d0039470 0x7f75d003b920 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f75cc006fd0 tx=0x7f75cc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:31.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.581+0000 7f75dd7fa640 1 -- 172.21.15.43:0/1195819817 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f75d403a6c0 con 0x7f75e0074c40 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.749+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f75e0106790 con 0x7f75e0074c40 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.752+0000 7f75dd7fa640 1 -- 172.21.15.43:0/1195819817 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f75d403f030 con 0x7f75e0074c40 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.754+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 >> v2:172.21.15.43:6800/225101565 conn(0x7f75d0039470 msgr2=0x7f75d003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.754+0000 7f75e7078640 1 --2- 172.21.15.43:0/1195819817 >> v2:172.21.15.43:6800/225101565 conn(0x7f75d0039470 0x7f75d003b920 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f75cc006fd0 tx=0x7f75cc006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.754+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 msgr2=0x7f75e01114d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.754+0000 7f75e7078640 1 --2- 172.21.15.43:0/1195819817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e01114d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75d4000c00 tx=0x7f75d4008040 comp rx=0 tx=0).stop 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.755+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 shutdown_connections 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.755+0000 7f75e7078640 1 --2- 172.21.15.43:0/1195819817 >> v2:172.21.15.43:6800/225101565 conn(0x7f75d0039470 0x7f75d003b920 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.755+0000 7f75e7078640 1 --2- 172.21.15.43:0/1195819817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e0074c40 0x7f75e01114d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.755+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 >> 172.21.15.43:0/1195819817 conn(0x7f75e00fabc0 msgr2=0x7f75e00fb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.755+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 shutdown_connections 2024-09-11T08:25:31.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:30.755+0000 7f75e7078640 1 -- 172.21.15.43:0/1195819817 wait complete. 2024-09-11T08:25:31.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:31 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:31.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:31 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:31.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1195819817' entity='client.admin' 2024-09-11T08:25:32.950 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.758+0000 7f2859987640 1 Processor -- start 2024-09-11T08:25:32.950 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.758+0000 7f2859987640 1 -- start start 2024-09-11T08:25:32.950 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.758+0000 7f2859987640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f28541031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:32.950 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.758+0000 7f2859987640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2854103730 con 0x7f28540754e0 2024-09-11T08:25:32.950 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.758+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f28541031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:32.950 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.758+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f28541031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46438/0 (socket says 172.21.15.43:46438) 2024-09-11T08:25:32.950 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.758+0000 7f2852ffd640 1 -- 172.21.15.43:0/2669399386 learned_addr learned my addr 172.21.15.43:0/2669399386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.759+0000 7f2852ffd640 1 -- 172.21.15.43:0/2669399386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854103870 con 0x7f28540754e0 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.759+0000 7f2852ffd640 1 --2- 172.21.15.43:0/2669399386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f28541031f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2844009840 tx=0x7f28440091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4c43b0c8b08b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2851ffb640 1 -- 172.21.15.43:0/2669399386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f284402add0 con 0x7f28540754e0 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2851ffb640 1 -- 172.21.15.43:0/2669399386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f28440328b0 con 0x7f28540754e0 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2851ffb640 1 -- 172.21.15.43:0/2669399386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2844031690 con 0x7f28540754e0 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2859987640 1 -- 172.21.15.43:0/2669399386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 msgr2=0x7f28541031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2859987640 1 --2- 172.21.15.43:0/2669399386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f28541031f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2844009840 tx=0x7f28440091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2859987640 1 -- 172.21.15.43:0/2669399386 shutdown_connections 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2859987640 1 --2- 172.21.15.43:0/2669399386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f28541031f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2859987640 1 -- 172.21.15.43:0/2669399386 >> 172.21.15.43:0/2669399386 conn(0x7f28540facc0 msgr2=0x7f28540fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2859987640 1 -- 172.21.15.43:0/2669399386 shutdown_connections 2024-09-11T08:25:32.951 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.760+0000 7f2859987640 1 -- 172.21.15.43:0/2669399386 wait complete. 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.761+0000 7f2859987640 1 Processor -- start 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.761+0000 7f2859987640 1 -- start start 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.761+0000 7f2859987640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f2854100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.761+0000 7f2859987640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2854101060 con 0x7f28540754e0 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.761+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f2854100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.761+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f2854100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46454/0 (socket says 172.21.15.43:46454) 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.761+0000 7f2852ffd640 1 -- 172.21.15.43:0/1175273486 learned_addr learned my addr 172.21.15.43:0/1175273486 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.761+0000 7f2852ffd640 1 -- 172.21.15.43:0/1175273486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2844009450 con 0x7f28540754e0 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.762+0000 7f2852ffd640 1 --2- 172.21.15.43:0/1175273486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f2854100b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2844006f00 tx=0x7f2844008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.762+0000 7f283b7fe640 1 -- 172.21.15.43:0/1175273486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2844031c60 con 0x7f28540754e0 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.762+0000 7f283b7fe640 1 -- 172.21.15.43:0/1175273486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f2844031dc0 con 0x7f28540754e0 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.762+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28540ff0f0 con 0x7f28540754e0 2024-09-11T08:25:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.762+0000 7f283b7fe640 1 -- 172.21.15.43:0/1175273486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f284403a540 con 0x7f28540754e0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.762+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28540ff5c0 con 0x7f28540754e0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.763+0000 7f283b7fe640 1 -- 172.21.15.43:0/1175273486 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f2844039070 con 0x7f28540754e0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.763+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285405ed70 con 0x7f28540754e0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.763+0000 7f283b7fe640 1 --2- 172.21.15.43:0/1175273486 >> v2:172.21.15.43:6800/225101565 conn(0x7f283c039430 0x7f283c03b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.763+0000 7f283b7fe640 1 -- 172.21.15.43:0/1175273486 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f284406cbd0 con 0x7f28540754e0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.763+0000 7f28527fc640 1 --2- 172.21.15.43:0/1175273486 >> v2:172.21.15.43:6800/225101565 conn(0x7f283c039430 0x7f283c03b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.764+0000 7f28527fc640 1 --2- 172.21.15.43:0/1175273486 >> v2:172.21.15.43:6800/225101565 conn(0x7f283c039430 0x7f283c03b8e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2848006fd0 tx=0x7f2848006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.768+0000 7f283b7fe640 1 -- 172.21.15.43:0/1175273486 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f2844044400 con 0x7f28540754e0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.938+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f285404b720 con 0x7f28540754e0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.941+0000 7f283b7fe640 1 -- 172.21.15.43:0/1175273486 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f284403f070 con 0x7f28540754e0 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 >> v2:172.21.15.43:6800/225101565 conn(0x7f283c039430 msgr2=0x7f283c03b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 --2- 172.21.15.43:0/1175273486 >> v2:172.21.15.43:6800/225101565 conn(0x7f283c039430 0x7f283c03b8e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2848006fd0 tx=0x7f2848006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:32.953 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 msgr2=0x7f2854100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 --2- 172.21.15.43:0/1175273486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f2854100b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2844006f00 tx=0x7f2844008040 comp rx=0 tx=0).stop 2024-09-11T08:25:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 shutdown_connections 2024-09-11T08:25:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 --2- 172.21.15.43:0/1175273486 >> v2:172.21.15.43:6800/225101565 conn(0x7f283c039430 0x7f283c03b8e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 --2- 172.21.15.43:0/1175273486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28540754e0 0x7f2854100b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 >> 172.21.15.43:0/1175273486 conn(0x7f28540facc0 msgr2=0x7f28540fb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 shutdown_connections 2024-09-11T08:25:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:31.943+0000 7f2859987640 1 -- 172.21.15.43:0/1175273486 wait complete. 2024-09-11T08:25:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-09-11T08:25:33.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1175273486' entity='client.admin' 2024-09-11T08:25:34.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:33 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:34.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:33 smithi043 ceph-mon[29355]: from='mgr.14118 172.21.15.43:0/3176390738' entity='mgr.y' 2024-09-11T08:25:34.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/949696168' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T08:25:34.948 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:34 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ignoring --setuser ceph since I am not root 2024-09-11T08:25:34.948 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:34 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ignoring --setgroup ceph since I am not root 2024-09-11T08:25:34.948 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:34 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:34.823+0000 7f7c82d6dec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:25:34.948 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:34 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:34.880+0000 7f7c82d6dec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:25:34.948 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:34 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:34.947+0000 7f7c82d6dec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:25:35.223 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:35 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:35.081+0000 7f7c82d6dec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:25:35.224 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:35 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:35.223+0000 7f7c82d6dec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:25:35.369 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.433+0000 7f8b7b1c2640 1 Processor -- start 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.433+0000 7f8b7b1c2640 1 -- start start 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.433+0000 7f8b7b1c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b741032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.433+0000 7f8b7b1c2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b74103830 con 0x7f8b74075420 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.433+0000 7f8b78f5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b741032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.433+0000 7f8b78f5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b741032f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55804/0 (socket says 172.21.15.43:55804) 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.433+0000 7f8b78f5e640 1 -- 172.21.15.43:0/2709810324 learned_addr learned my addr 172.21.15.43:0/2709810324 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.434+0000 7f8b78f5e640 1 -- 172.21.15.43:0/2709810324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b74103970 con 0x7f8b74075420 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.434+0000 7f8b78f5e640 1 --2- 172.21.15.43:0/2709810324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b741032f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009840 tx=0x7f8b680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d591ade7481ef29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.435+0000 7f8b737fe640 1 -- 172.21.15.43:0/2709810324 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b6802add0 con 0x7f8b74075420 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.435+0000 7f8b737fe640 1 -- 172.21.15.43:0/2709810324 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8b680328b0 con 0x7f8b74075420 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.435+0000 7f8b737fe640 1 -- 172.21.15.43:0/2709810324 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b68031690 con 0x7f8b74075420 2024-09-11T08:25:35.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/2709810324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 msgr2=0x7f8b741032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 --2- 172.21.15.43:0/2709810324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b741032f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009840 tx=0x7f8b680091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/2709810324 shutdown_connections 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 --2- 172.21.15.43:0/2709810324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b741032f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/2709810324 >> 172.21.15.43:0/2709810324 conn(0x7f8b740fad60 msgr2=0x7f8b740fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/2709810324 shutdown_connections 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/2709810324 wait complete. 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 Processor -- start 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.436+0000 7f8b7b1c2640 1 -- start start 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.437+0000 7f8b7b1c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b74100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.437+0000 7f8b7b1c2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b74101160 con 0x7f8b74075420 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.437+0000 7f8b78f5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b74100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.437+0000 7f8b78f5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b74100c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55816/0 (socket says 172.21.15.43:55816) 2024-09-11T08:25:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.437+0000 7f8b78f5e640 1 -- 172.21.15.43:0/949696168 learned_addr learned my addr 172.21.15.43:0/949696168 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.437+0000 7f8b78f5e640 1 -- 172.21.15.43:0/949696168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68009450 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.437+0000 7f8b78f5e640 1 --2- 172.21.15.43:0/949696168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b74100c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006f00 tx=0x7f8b68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.438+0000 7f8b717fa640 1 -- 172.21.15.43:0/949696168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b68031c60 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.438+0000 7f8b717fa640 1 -- 172.21.15.43:0/949696168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8b68031dc0 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.438+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b740ff1f0 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.438+0000 7f8b717fa640 1 -- 172.21.15.43:0/949696168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b6803a540 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.438+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b740ff660 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.439+0000 7f8b717fa640 1 -- 172.21.15.43:0/949696168 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f8b68039070 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.439+0000 7f8b717fa640 1 --2- 172.21.15.43:0/949696168 >> v2:172.21.15.43:6800/225101565 conn(0x7f8b64039470 0x7f8b6403b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.439+0000 7f8b717fa640 1 -- 172.21.15.43:0/949696168 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f8b6806ca40 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.439+0000 7f8b73fff640 1 --2- 172.21.15.43:0/949696168 >> v2:172.21.15.43:6800/225101565 conn(0x7f8b64039470 0x7f8b6403b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.439+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7404b720 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.440+0000 7f8b73fff640 1 --2- 172.21.15.43:0/949696168 >> v2:172.21.15.43:6800/225101565 conn(0x7f8b64039470 0x7f8b6403b920 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b60006fd0 tx=0x7f8b60006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.444+0000 7f8b717fa640 1 -- 172.21.15.43:0/949696168 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8b6803f020 con 0x7f8b74075420 2024-09-11T08:25:35.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:33.691+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8b740ff7f0 con 0x7f8b74075420 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.681+0000 7f8b717fa640 1 -- 172.21.15.43:0/949696168 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 9) v1 ==== 45470+0+0 (secure 0 0 0) 0x7f8b6803a6a0 con 0x7f8b74075420 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.681+0000 7f8b717fa640 1 -- 172.21.15.43:0/949696168 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f8b6806b0e0 con 0x7f8b74075420 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.683+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 >> v2:172.21.15.43:6800/225101565 conn(0x7f8b64039470 msgr2=0x7f8b6403b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.683+0000 7f8b7b1c2640 1 --2- 172.21.15.43:0/949696168 >> v2:172.21.15.43:6800/225101565 conn(0x7f8b64039470 0x7f8b6403b920 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b60006fd0 tx=0x7f8b60006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.683+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 msgr2=0x7f8b74100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.683+0000 7f8b7b1c2640 1 --2- 172.21.15.43:0/949696168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b74100c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006f00 tx=0x7f8b68008040 comp rx=0 tx=0).stop 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.684+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 shutdown_connections 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.684+0000 7f8b7b1c2640 1 --2- 172.21.15.43:0/949696168 >> v2:172.21.15.43:6800/225101565 conn(0x7f8b64039470 0x7f8b6403b920 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.684+0000 7f8b7b1c2640 1 --2- 172.21.15.43:0/949696168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b74075420 0x7f8b74100c20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.684+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 >> 172.21.15.43:0/949696168 conn(0x7f8b740fad60 msgr2=0x7f8b740fb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.684+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 shutdown_connections 2024-09-11T08:25:35.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:34.684+0000 7f8b7b1c2640 1 -- 172.21.15.43:0/949696168 wait complete. 2024-09-11T08:25:35.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/949696168' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T08:25:35.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:35 smithi043 ceph-mon[29355]: mgrmap e9: y(active, since 19s) 2024-09-11T08:25:35.743 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:35 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:35.496+0000 7f7c82d6dec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:25:35.743 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:35 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:35.559+0000 7f7c82d6dec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:25:36.084 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:35 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:35.742+0000 7f7c82d6dec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:25:36.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3543243370' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:25:36.980 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:36 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:36.648+0000 7f7c82d6dec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:25:36.980 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:36 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:36.716+0000 7f7c82d6dec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:25:36.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.164+0000 7efebd48d640 1 Processor -- start 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.164+0000 7efebd48d640 1 -- start start 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.165+0000 7efebd48d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8100e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.165+0000 7efebd48d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8101340 con 0x7efeb81009f0 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.165+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:36.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.165+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8100e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55844/0 (socket says 172.21.15.43:55844) 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.165+0000 7efeb6ffd640 1 -- 172.21.15.43:0/170454337 learned_addr learned my addr 172.21.15.43:0/170454337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.165+0000 7efeb6ffd640 1 -- 172.21.15.43:0/170454337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8101480 con 0x7efeb81009f0 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.166+0000 7efeb6ffd640 1 --2- 172.21.15.43:0/170454337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8100e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efea8009e50 tx=0x7efea8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95718627ca79a466 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.166+0000 7efeb5ffb640 1 -- 172.21.15.43:0/170454337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efea8031400 con 0x7efeb81009f0 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.166+0000 7efeb5ffb640 1 -- 172.21.15.43:0/170454337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7efea802d050 con 0x7efeb81009f0 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.166+0000 7efeb5ffb640 1 -- 172.21.15.43:0/170454337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efea8007690 con 0x7efeb81009f0 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 -- 172.21.15.43:0/170454337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 msgr2=0x7efeb8100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 --2- 172.21.15.43:0/170454337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8100e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efea8009e50 tx=0x7efea8009840 comp rx=0 tx=0).stop 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 -- 172.21.15.43:0/170454337 shutdown_connections 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 --2- 172.21.15.43:0/170454337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8100e00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 -- 172.21.15.43:0/170454337 >> 172.21.15.43:0/170454337 conn(0x7efeb80fa340 msgr2=0x7efeb80fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 -- 172.21.15.43:0/170454337 shutdown_connections 2024-09-11T08:25:36.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 -- 172.21.15.43:0/170454337 wait complete. 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 Processor -- start 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.167+0000 7efebd48d640 1 -- start start 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efebd48d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efebd48d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8111ac0 con 0x7efeb81009f0 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55852/0 (socket says 172.21.15.43:55852) 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efeb6ffd640 1 -- 172.21.15.43:0/3543243370 learned_addr learned my addr 172.21.15.43:0/3543243370 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efeb6ffd640 1 -- 172.21.15.43:0/3543243370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8009ae0 con 0x7efeb81009f0 2024-09-11T08:25:36.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efeb6ffd640 1 --2- 172.21.15.43:0/3543243370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8111580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efea8031a00 tx=0x7efea8031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efeaf7fe640 1 -- 172.21.15.43:0/3543243370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efea8008930 con 0x7efeb81009f0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efeaf7fe640 1 -- 172.21.15.43:0/3543243370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7efea8008f00 con 0x7efeb81009f0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.168+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb8111cc0 con 0x7efeb81009f0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.169+0000 7efeaf7fe640 1 -- 172.21.15.43:0/3543243370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efea803bc10 con 0x7efeb81009f0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.169+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb8112130 con 0x7efeb81009f0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.169+0000 7efeaf7fe640 1 -- 172.21.15.43:0/3543243370 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 45470+0+0 (secure 0 0 0) 0x7efea8038050 con 0x7efeb81009f0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.170+0000 7efeaf7fe640 1 --2- 172.21.15.43:0/3543243370 >> v2:172.21.15.43:6800/225101565 conn(0x7efea40394c0 0x7efea403b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.170+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb804b720 con 0x7efeb81009f0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.170+0000 7efeaf7fe640 1 -- 172.21.15.43:0/3543243370 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7efea806d6a0 con 0x7efeb81009f0 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.170+0000 7efeb67fc640 1 -- 172.21.15.43:0/3543243370 >> v2:172.21.15.43:6800/225101565 conn(0x7efea40394c0 msgr2=0x7efea403b970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/225101565 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.170+0000 7efeb67fc640 1 --2- 172.21.15.43:0/3543243370 >> v2:172.21.15.43:6800/225101565 conn(0x7efea40394c0 0x7efea403b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:25:36.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.176+0000 7efeaf7fe640 1 -- 172.21.15.43:0/3543243370 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7efea8036070 con 0x7efeb81009f0 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.371+0000 7efeb67fc640 1 -- 172.21.15.43:0/3543243370 >> v2:172.21.15.43:6800/225101565 conn(0x7efea40394c0 msgr2=0x7efea403b970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/225101565 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.371+0000 7efeb67fc640 1 --2- 172.21.15.43:0/3543243370 >> v2:172.21.15.43:6800/225101565 conn(0x7efea40394c0 0x7efea403b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.386+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7efeb8112130 con 0x7efeb81009f0 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.387+0000 7efeaf7fe640 1 -- 172.21.15.43:0/3543243370 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7efea8041030 con 0x7efeb81009f0 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 >> v2:172.21.15.43:6800/225101565 conn(0x7efea40394c0 msgr2=0x7efea403b970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 --2- 172.21.15.43:0/3543243370 >> v2:172.21.15.43:6800/225101565 conn(0x7efea40394c0 0x7efea403b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 msgr2=0x7efeb8111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 --2- 172.21.15.43:0/3543243370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8111580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efea8031a00 tx=0x7efea8031f90 comp rx=0 tx=0).stop 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 shutdown_connections 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 --2- 172.21.15.43:0/3543243370 >> v2:172.21.15.43:6800/225101565 conn(0x7efea40394c0 0x7efea403b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 --2- 172.21.15.43:0/3543243370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb81009f0 0x7efeb8111580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:36.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 >> 172.21.15.43:0/3543243370 conn(0x7efeb80fa340 msgr2=0x7efeb80fb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:36.993 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 shutdown_connections 2024-09-11T08:25:36.993 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:36.389+0000 7efebd48d640 1 -- 172.21.15.43:0/3543243370 wait complete. 2024-09-11T08:25:36.993 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-11T08:25:36.993 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 9... 2024-09-11T08:25:37.588 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:37.472+0000 7f7c82d6dec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:25:37.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:37.587+0000 7f7c82d6dec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:25:37.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:37.713+0000 7f7c82d6dec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:25:37.895 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:37.775+0000 7f7c82d6dec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:25:38.175 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:37.905+0000 7f7c82d6dec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:25:38.175 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:37.975+0000 7f7c82d6dec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:25:38.175 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:38.035+0000 7f7c82d6dec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:25:38.175 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:38.175+0000 7f7c82d6dec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:25:38.452 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:38.241+0000 7f7c82d6dec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:25:38.452 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:38.308+0000 7f7c82d6dec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:25:38.452 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:38.392+0000 7f7c82d6dec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:25:38.453 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:38.452+0000 7f7c82d6dec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:25:38.836 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:38.511+0000 7f7c82d6dec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:25:39.540 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:39.287+0000 7f7c82d6dec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:25:39.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: Active manager daemon y restarted 2024-09-11T08:25:39.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: Activating manager daemon y 2024-09-11T08:25:39.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T08:25:39.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: mgrmap e10: y(active, starting, since 0.00328959s) 2024-09-11T08:25:39.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:25:39.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:25:39.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:25:39.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:25:39.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:25:39.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: Manager daemon y is now available 2024-09-11T08:25:39.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:39.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:25:39.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:25:41.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:25:41.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T08:25:41.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:25:41.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:25:41.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.428+0000 7fc93fc21640 1 Processor -- start 2024-09-11T08:25:41.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.428+0000 7fc93fc21640 1 -- start start 2024-09-11T08:25:41.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.428+0000 7fc93fc21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938103240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.428+0000 7fc93fc21640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc938075800 con 0x7fc9380753f0 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.429+0000 7fc93d9bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.429+0000 7fc93d9bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55864/0 (socket says 172.21.15.43:55864) 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.429+0000 7fc93d9bd640 1 -- 172.21.15.43:0/2717892540 learned_addr learned my addr 172.21.15.43:0/2717892540 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.429+0000 7fc93d9bd640 1 -- 172.21.15.43:0/2717892540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938103780 con 0x7fc9380753f0 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93d9bd640 1 --2- 172.21.15.43:0/2717892540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938103240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc934009840 tx=0x7fc9340091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32e4fd67e409c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93c9bb640 1 -- 172.21.15.43:0/2717892540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc93402add0 con 0x7fc9380753f0 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93c9bb640 1 -- 172.21.15.43:0/2717892540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fc9340328b0 con 0x7fc9380753f0 2024-09-11T08:25:41.146 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93c9bb640 1 -- 172.21.15.43:0/2717892540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc934031690 con 0x7fc9380753f0 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93fc21640 1 -- 172.21.15.43:0/2717892540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 msgr2=0x7fc938103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93fc21640 1 --2- 172.21.15.43:0/2717892540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938103240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc934009840 tx=0x7fc9340091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93fc21640 1 -- 172.21.15.43:0/2717892540 shutdown_connections 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93fc21640 1 --2- 172.21.15.43:0/2717892540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938103240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.430+0000 7fc93fc21640 1 -- 172.21.15.43:0/2717892540 >> 172.21.15.43:0/2717892540 conn(0x7fc9380fac50 msgr2=0x7fc9380fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93fc21640 1 -- 172.21.15.43:0/2717892540 shutdown_connections 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93fc21640 1 -- 172.21.15.43:0/2717892540 wait complete. 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93fc21640 1 Processor -- start 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93fc21640 1 -- start start 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93fc21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938100b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93fc21640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9381010b0 con 0x7fc9380753f0 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93d9bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938100b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:41.147 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93d9bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938100b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55878/0 (socket says 172.21.15.43:55878) 2024-09-11T08:25:41.148 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.431+0000 7fc93d9bd640 1 -- 172.21.15.43:0/2338480802 learned_addr learned my addr 172.21.15.43:0/2338480802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:41.148 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.432+0000 7fc93d9bd640 1 -- 172.21.15.43:0/2338480802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc934009450 con 0x7fc9380753f0 2024-09-11T08:25:41.148 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.432+0000 7fc93d9bd640 1 --2- 172.21.15.43:0/2338480802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938100b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc934006f00 tx=0x7fc934008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:41.148 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.432+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc934031c60 con 0x7fc9380753f0 2024-09-11T08:25:41.148 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.432+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fc934031dc0 con 0x7fc9380753f0 2024-09-11T08:25:41.148 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.432+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9380ff140 con 0x7fc9380753f0 2024-09-11T08:25:41.148 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.432+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9380ff5b0 con 0x7fc9380753f0 2024-09-11T08:25:41.149 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.432+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc93403a540 con 0x7fc9380753f0 2024-09-11T08:25:41.149 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.433+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 45470+0+0 (secure 0 0 0) 0x7fc934039070 con 0x7fc9380753f0 2024-09-11T08:25:41.149 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.433+0000 7fc92a7fc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 0x7fc92403b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:41.149 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.433+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 --> v2:172.21.15.43:6800/225101565 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc92403bff0 con 0x7fc924039470 2024-09-11T08:25:41.149 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.433+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fc93406cc90 con 0x7fc9380753f0 2024-09-11T08:25:41.150 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.433+0000 7fc93d1bc640 1 -- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 msgr2=0x7fc92403b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/225101565 2024-09-11T08:25:41.150 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.433+0000 7fc93d1bc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 0x7fc92403b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:25:41.150 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.634+0000 7fc93d1bc640 1 -- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 msgr2=0x7fc92403b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/225101565 2024-09-11T08:25:41.150 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:37.634+0000 7fc93d1bc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 0x7fc92403b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T08:25:41.150 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:38.035+0000 7fc93d1bc640 1 -- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 msgr2=0x7fc92403b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/225101565 2024-09-11T08:25:41.150 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:38.035+0000 7fc93d1bc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 0x7fc92403b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T08:25:41.151 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:38.836+0000 7fc93d1bc640 1 -- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 msgr2=0x7fc92403b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/225101565 2024-09-11T08:25:41.151 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:38.836+0000 7fc93d1bc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 0x7fc92403b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T08:25:41.151 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:39.291+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 10) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fc93406bd20 con 0x7fc9380753f0 2024-09-11T08:25:41.151 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:39.291+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 msgr2=0x7fc92403b920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:25:41.151 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:39.291+0000 7fc92a7fc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 0x7fc92403b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:41.151 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.293+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 11) v1 ==== 45364+0+0 (secure 0 0 0) 0x7fc93406e450 con 0x7fc9380753f0 2024-09-11T08:25:41.152 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.293+0000 7fc92a7fc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc92403cef0 0x7fc92403f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:41.152 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.293+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 --> v2:172.21.15.43:6800/1424089977 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc92403bff0 con 0x7fc92403cef0 2024-09-11T08:25:41.152 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.294+0000 7fc93d1bc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc92403cef0 0x7fc92403f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:41.152 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.294+0000 7fc93d1bc640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc92403cef0 0x7fc92403f2d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc92c003940 tx=0x7fc92c009350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:41.152 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.295+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fc92403bff0 con 0x7fc92403cef0 2024-09-11T08:25:41.153 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 --> v2:172.21.15.43:6800/1424089977 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc93804b720 con 0x7fc92403cef0 2024-09-11T08:25:41.153 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc92a7fc640 1 -- 172.21.15.43:0/2338480802 <== mgr.14150 v2:172.21.15.43:6800/1424089977 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc93804b720 con 0x7fc92403cef0 2024-09-11T08:25:41.153 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc92403cef0 msgr2=0x7fc92403f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:41.153 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc93fc21640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc92403cef0 0x7fc92403f2d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc92c003940 tx=0x7fc92c009350 comp rx=0 tx=0).stop 2024-09-11T08:25:41.153 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 msgr2=0x7fc938100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:41.154 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc93fc21640 1 --2- 172.21.15.43:0/2338480802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938100b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc934006f00 tx=0x7fc934008040 comp rx=0 tx=0).stop 2024-09-11T08:25:41.154 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 shutdown_connections 2024-09-11T08:25:41.154 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc93fc21640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc92403cef0 0x7fc92403f2d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:41.154 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.300+0000 7fc93fc21640 1 --2- 172.21.15.43:0/2338480802 >> v2:172.21.15.43:6800/225101565 conn(0x7fc924039470 0x7fc92403b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:41.154 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.301+0000 7fc93fc21640 1 --2- 172.21.15.43:0/2338480802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380753f0 0x7fc938100b70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:41.154 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.301+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 >> 172.21.15.43:0/2338480802 conn(0x7fc9380fac50 msgr2=0x7fc9380fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:41.155 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.301+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 shutdown_connections 2024-09-11T08:25:41.155 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:40.301+0000 7fc93fc21640 1 -- 172.21.15.43:0/2338480802 wait complete. 2024-09-11T08:25:41.155 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 9 is available 2024-09-11T08:25:41.155 INFO:teuthology.orchestra.run.smithi043.stdout:Generating a dashboard self-signed certificate... 2024-09-11T08:25:41.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-mon[29355]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T08:25:41.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-mon[29355]: mgrmap e11: y(active, since 1.00609s) 2024-09-11T08:25:41.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-mon[29355]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T08:25:41.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:41.440+0000 7f7c338ca640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:25:41] ENGINE Error in HTTPServer.serve 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: Traceback (most recent call last): 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._connections.run(self.expiration_interval) 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._run(expiration_interval) 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:25:41.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: s = self.context.wrap_socket( 2024-09-11T08:25:41.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:25:41.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: return self.sslsocket_class._create( 2024-09-11T08:25:41.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:25:41.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self.do_handshake() 2024-09-11T08:25:41.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:25:41.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._sslobj.do_handshake() 2024-09-11T08:25:41.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:25:41.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:25:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:25:42.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.562+0000 7fed7cbea640 1 Processor -- start 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.562+0000 7fed7cbea640 1 -- start start 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.562+0000 7fed7cbea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed78100e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.562+0000 7fed7cbea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed78101370 con 0x7fed78100a20 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.562+0000 7fed7659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed78100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.562+0000 7fed7659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed78100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55934/0 (socket says 172.21.15.43:55934) 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.562+0000 7fed7659c640 1 -- 172.21.15.43:0/2053443540 learned_addr learned my addr 172.21.15.43:0/2053443540 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.563+0000 7fed7659c640 1 -- 172.21.15.43:0/2053443540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed781014b0 con 0x7fed78100a20 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.564+0000 7fed7659c640 1 --2- 172.21.15.43:0/2053443540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed78100e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed68009e50 tx=0x7fed68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ab3ef2aeb7eabe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.564+0000 7fed7559a640 1 -- 172.21.15.43:0/2053443540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed68031400 con 0x7fed78100a20 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.564+0000 7fed7559a640 1 -- 172.21.15.43:0/2053443540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fed6802d050 con 0x7fed78100a20 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.564+0000 7fed7559a640 1 -- 172.21.15.43:0/2053443540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed68007690 con 0x7fed78100a20 2024-09-11T08:25:42.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.564+0000 7fed7cbea640 1 -- 172.21.15.43:0/2053443540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 msgr2=0x7fed78100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.564+0000 7fed7cbea640 1 --2- 172.21.15.43:0/2053443540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed78100e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed68009e50 tx=0x7fed68009840 comp rx=0 tx=0).stop 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.565+0000 7fed7cbea640 1 -- 172.21.15.43:0/2053443540 shutdown_connections 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.565+0000 7fed7cbea640 1 --2- 172.21.15.43:0/2053443540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed78100e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.565+0000 7fed7cbea640 1 -- 172.21.15.43:0/2053443540 >> 172.21.15.43:0/2053443540 conn(0x7fed780fa340 msgr2=0x7fed780fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.565+0000 7fed7cbea640 1 -- 172.21.15.43:0/2053443540 shutdown_connections 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.565+0000 7fed7cbea640 1 -- 172.21.15.43:0/2053443540 wait complete. 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.565+0000 7fed7cbea640 1 Processor -- start 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.565+0000 7fed7cbea640 1 -- start start 2024-09-11T08:25:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.565+0000 7fed7cbea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed780ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed7cbea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed78111fc0 con 0x7fed78100a20 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed7659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed780ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed7659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed780ff1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55936/0 (socket says 172.21.15.43:55936) 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed7659c640 1 -- 172.21.15.43:0/825049552 learned_addr learned my addr 172.21.15.43:0/825049552 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed7659c640 1 -- 172.21.15.43:0/825049552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed68009ae0 con 0x7fed78100a20 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed7659c640 1 --2- 172.21.15.43:0/825049552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed780ff1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed68003400 tx=0x7fed68031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed66ffd640 1 -- 172.21.15.43:0/825049552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed68031880 con 0x7fed78100a20 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed66ffd640 1 -- 172.21.15.43:0/825049552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fed6803b410 con 0x7fed78100a20 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed78112100 con 0x7fed78100a20 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed66ffd640 1 -- 172.21.15.43:0/825049552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed680433c0 con 0x7fed78100a20 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.566+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed78112570 con 0x7fed78100a20 2024-09-11T08:25:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.567+0000 7fed66ffd640 1 -- 172.21.15.43:0/825049552 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 11) v1 ==== 45364+0+0 (secure 0 0 0) 0x7fed68038070 con 0x7fed78100a20 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.567+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7804b720 con 0x7fed78100a20 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.568+0000 7fed66ffd640 1 --2- 172.21.15.43:0/825049552 >> v2:172.21.15.43:6800/1424089977 conn(0x7fed600393b0 0x7fed6003b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.568+0000 7fed66ffd640 1 -- 172.21.15.43:0/825049552 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fed6806c660 con 0x7fed78100a20 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.568+0000 7fed75d9b640 1 --2- 172.21.15.43:0/825049552 >> v2:172.21.15.43:6800/1424089977 conn(0x7fed600393b0 0x7fed6003b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.568+0000 7fed75d9b640 1 --2- 172.21.15.43:0/825049552 >> v2:172.21.15.43:6800/1424089977 conn(0x7fed600393b0 0x7fed6003b860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed6c006fd0 tx=0x7fed6c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.575+0000 7fed66ffd640 1 -- 172.21.15.43:0/825049552 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fed6804b530 con 0x7fed78100a20 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.626+0000 7fed66ffd640 1 -- 172.21.15.43:0/825049552 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 12) v1 ==== 45470+0+0 (secure 0 0 0) 0x7fed68038cf0 con 0x7fed78100a20 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.750+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fed780fb5a0 con 0x7fed600393b0 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.808+0000 7fed66ffd640 1 -- 172.21.15.43:0/825049552 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fed780fb5a0 con 0x7fed600393b0 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 >> v2:172.21.15.43:6800/1424089977 conn(0x7fed600393b0 msgr2=0x7fed6003b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 --2- 172.21.15.43:0/825049552 >> v2:172.21.15.43:6800/1424089977 conn(0x7fed600393b0 0x7fed6003b860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed6c006fd0 tx=0x7fed6c006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 msgr2=0x7fed780ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:42.382 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 --2- 172.21.15.43:0/825049552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed780ff1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed68003400 tx=0x7fed68031f90 comp rx=0 tx=0).stop 2024-09-11T08:25:42.383 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 shutdown_connections 2024-09-11T08:25:42.383 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 --2- 172.21.15.43:0/825049552 >> v2:172.21.15.43:6800/1424089977 conn(0x7fed600393b0 0x7fed6003b860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:42.383 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 --2- 172.21.15.43:0/825049552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed78100a20 0x7fed780ff1f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:42.383 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 >> 172.21.15.43:0/825049552 conn(0x7fed780fa340 msgr2=0x7fed780fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:42.383 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 shutdown_connections 2024-09-11T08:25:42.383 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:41.810+0000 7fed7cbea640 1 -- 172.21.15.43:0/825049552 wait complete. 2024-09-11T08:25:42.383 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-09-11T08:25:42.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: [11/Sep/2024:08:25:41] ENGINE Bus STARTING 2024-09-11T08:25:42.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: [11/Sep/2024:08:25:41] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T08:25:42.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: [11/Sep/2024:08:25:41] ENGINE Bus STARTED 2024-09-11T08:25:42.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: [11/Sep/2024:08:25:41] ENGINE Error in HTTPServer.serve 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:25:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: mgrmap e12: y(active, since 2s) 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:42 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:43 smithi043 ceph-mon[29355]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:43 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:44 smithi043 ceph-mon[29355]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:44.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$C.21TkgTyhzzSQz89TD/ZefwChqTpzMYeAZ2ItQrdVAkiYVOyG4OC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726043143, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T08:25:44.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.814+0000 7fc2da482640 1 Processor -- start 2024-09-11T08:25:44.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.814+0000 7fc2da482640 1 -- start start 2024-09-11T08:25:44.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.815+0000 7fc2da482640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d41009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.815+0000 7fc2da482640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d4100f00 con 0x7fc2d41005b0 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.815+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d41009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.815+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d41009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37870/0 (socket says 172.21.15.43:37870) 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.815+0000 7fc2d3fff640 1 -- 172.21.15.43:0/302963150 learned_addr learned my addr 172.21.15.43:0/302963150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.815+0000 7fc2d3fff640 1 -- 172.21.15.43:0/302963150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d4101040 con 0x7fc2d41005b0 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.816+0000 7fc2d3fff640 1 --2- 172.21.15.43:0/302963150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d41009c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009840 tx=0x7fc2c40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ead5f4a35f51d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.816+0000 7fc2d2ffd640 1 -- 172.21.15.43:0/302963150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2c402add0 con 0x7fc2d41005b0 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.816+0000 7fc2d2ffd640 1 -- 172.21.15.43:0/302963150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fc2c40328b0 con 0x7fc2d41005b0 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.816+0000 7fc2d2ffd640 1 -- 172.21.15.43:0/302963150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2c4031690 con 0x7fc2d41005b0 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 -- 172.21.15.43:0/302963150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 msgr2=0x7fc2d41009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 --2- 172.21.15.43:0/302963150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d41009c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009840 tx=0x7fc2c40091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:44.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 -- 172.21.15.43:0/302963150 shutdown_connections 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 --2- 172.21.15.43:0/302963150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d41009c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 -- 172.21.15.43:0/302963150 >> 172.21.15.43:0/302963150 conn(0x7fc2d40fbe10 msgr2=0x7fc2d40fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 -- 172.21.15.43:0/302963150 shutdown_connections 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 -- 172.21.15.43:0/302963150 wait complete. 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 Processor -- start 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.817+0000 7fc2da482640 1 -- start start 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2da482640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d410f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2da482640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d410f7a0 con 0x7fc2d41005b0 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d410f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d410f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37886/0 (socket says 172.21.15.43:37886) 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2d3fff640 1 -- 172.21.15.43:0/1650393341 learned_addr learned my addr 172.21.15.43:0/1650393341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2d3fff640 1 -- 172.21.15.43:0/1650393341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c4009450 con 0x7fc2d41005b0 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2d3fff640 1 --2- 172.21.15.43:0/1650393341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d410f260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4031450 tx=0x7fc2c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:44.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2d0ff9640 1 -- 172.21.15.43:0/1650393341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2c4031ca0 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2d0ff9640 1 -- 172.21.15.43:0/1650393341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fc2c4031e00 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d410f9a0 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2d0ff9640 1 -- 172.21.15.43:0/1650393341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2c403a540 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.818+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d410fe10 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.819+0000 7fc2d0ff9640 1 -- 172.21.15.43:0/1650393341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) v1 ==== 45470+0+0 (secure 0 0 0) 0x7fc2c4039070 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.819+0000 7fc2d0ff9640 1 --2- 172.21.15.43:0/1650393341 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc2bc0394c0 0x7fc2bc03b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.819+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d404b720 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.819+0000 7fc2d0ff9640 1 -- 172.21.15.43:0/1650393341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc2c406c940 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.820+0000 7fc2d37fe640 1 --2- 172.21.15.43:0/1650393341 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc2bc0394c0 0x7fc2bc03b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.820+0000 7fc2d37fe640 1 --2- 172.21.15.43:0/1650393341 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc2bc0394c0 0x7fc2bc03b970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8006fd0 tx=0x7fc2c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:42.824+0000 7fc2d0ff9640 1 -- 172.21.15.43:0/1650393341 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc2c403a740 con 0x7fc2d41005b0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.005+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc2d405eb60 con 0x7fc2bc0394c0 2024-09-11T08:25:44.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.253+0000 7fc2d0ff9640 1 -- 172.21.15.43:0/1650393341 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fc2d405eb60 con 0x7fc2bc0394c0 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.255+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc2bc0394c0 msgr2=0x7fc2bc03b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.255+0000 7fc2da482640 1 --2- 172.21.15.43:0/1650393341 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc2bc0394c0 0x7fc2bc03b970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8006fd0 tx=0x7fc2c8006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.255+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 msgr2=0x7fc2d410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.255+0000 7fc2da482640 1 --2- 172.21.15.43:0/1650393341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d410f260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4031450 tx=0x7fc2c4008040 comp rx=0 tx=0).stop 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.255+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 shutdown_connections 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.255+0000 7fc2da482640 1 --2- 172.21.15.43:0/1650393341 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc2bc0394c0 0x7fc2bc03b970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.255+0000 7fc2da482640 1 --2- 172.21.15.43:0/1650393341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d41005b0 0x7fc2d410f260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.255+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 >> 172.21.15.43:0/1650393341 conn(0x7fc2d40fbe10 msgr2=0x7fc2d40fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.256+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 shutdown_connections 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:43.256+0000 7fc2da482640 1 -- 172.21.15.43:0/1650393341 wait complete. 2024-09-11T08:25:44.503 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-09-11T08:25:45.641 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/964054933' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T08:25:45.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T08:25:45.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.958+0000 7f816f7bb640 1 Processor -- start 2024-09-11T08:25:45.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.959+0000 7f816f7bb640 1 -- start start 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.959+0000 7f816f7bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f81681009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.959+0000 7f816f7bb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8168100f00 con 0x7f81681005b0 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.959+0000 7f816d557640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f81681009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.959+0000 7f816d557640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f81681009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37892/0 (socket says 172.21.15.43:37892) 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.959+0000 7f816d557640 1 -- 172.21.15.43:0/28849859 learned_addr learned my addr 172.21.15.43:0/28849859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.959+0000 7f816d557640 1 -- 172.21.15.43:0/28849859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8168101040 con 0x7f81681005b0 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.960+0000 7f816d557640 1 --2- 172.21.15.43:0/28849859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f81681009c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8164009840 tx=0x7f81640091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ef8512aa57e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.960+0000 7f815ffff640 1 -- 172.21.15.43:0/28849859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f816402add0 con 0x7f81681005b0 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.960+0000 7f815ffff640 1 -- 172.21.15.43:0/28849859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f81640328b0 con 0x7f81681005b0 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.960+0000 7f815ffff640 1 -- 172.21.15.43:0/28849859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8164031690 con 0x7f81681005b0 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.961+0000 7f816f7bb640 1 -- 172.21.15.43:0/28849859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 msgr2=0x7f81681009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.961+0000 7f816f7bb640 1 --2- 172.21.15.43:0/28849859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f81681009c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8164009840 tx=0x7f81640091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:45.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.961+0000 7f816f7bb640 1 -- 172.21.15.43:0/28849859 shutdown_connections 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.961+0000 7f816f7bb640 1 --2- 172.21.15.43:0/28849859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f81681009c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.961+0000 7f816f7bb640 1 -- 172.21.15.43:0/28849859 >> 172.21.15.43:0/28849859 conn(0x7f81680fbe10 msgr2=0x7f81680fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.961+0000 7f816f7bb640 1 -- 172.21.15.43:0/28849859 shutdown_connections 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.961+0000 7f816f7bb640 1 -- 172.21.15.43:0/28849859 wait complete. 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.961+0000 7f816f7bb640 1 Processor -- start 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816f7bb640 1 -- start start 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816f7bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f816810f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816f7bb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816810f7c0 con 0x7f81681005b0 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816d557640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f816810f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816d557640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f816810f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37900/0 (socket says 172.21.15.43:37900) 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816d557640 1 -- 172.21.15.43:0/964054933 learned_addr learned my addr 172.21.15.43:0/964054933 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816d557640 1 -- 172.21.15.43:0/964054933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8164009450 con 0x7f81681005b0 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816d557640 1 --2- 172.21.15.43:0/964054933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f816810f280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8164006f00 tx=0x7f8164008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:45.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f815dffb640 1 -- 172.21.15.43:0/964054933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8164031c60 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f815dffb640 1 -- 172.21.15.43:0/964054933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8164031dc0 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816810f9c0 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.962+0000 7f815dffb640 1 -- 172.21.15.43:0/964054933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f816403a540 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.963+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816810fe30 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.964+0000 7f815dffb640 1 -- 172.21.15.43:0/964054933 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) v1 ==== 45470+0+0 (secure 0 0 0) 0x7f8164039070 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.964+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816804b720 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.964+0000 7f815dffb640 1 --2- 172.21.15.43:0/964054933 >> v2:172.21.15.43:6800/1424089977 conn(0x7f81540394c0 0x7f815403b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.964+0000 7f815dffb640 1 -- 172.21.15.43:0/964054933 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f816406c870 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.964+0000 7f816cd56640 1 --2- 172.21.15.43:0/964054933 >> v2:172.21.15.43:6800/1424089977 conn(0x7f81540394c0 0x7f815403b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.964+0000 7f816cd56640 1 --2- 172.21.15.43:0/964054933 >> v2:172.21.15.43:6800/1424089977 conn(0x7f81540394c0 0x7f815403b970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8158006fd0 tx=0x7f8158006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:44.969+0000 7f815dffb640 1 -- 172.21.15.43:0/964054933 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f816403b440 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.132+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f81681023c0 con 0x7f81681005b0 2024-09-11T08:25:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.132+0000 7f815dffb640 1 -- 172.21.15.43:0/964054933 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f816403f020 con 0x7f81681005b0 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 >> v2:172.21.15.43:6800/1424089977 conn(0x7f81540394c0 msgr2=0x7f815403b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 --2- 172.21.15.43:0/964054933 >> v2:172.21.15.43:6800/1424089977 conn(0x7f81540394c0 0x7f815403b970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8158006fd0 tx=0x7f8158006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 msgr2=0x7f816810f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 --2- 172.21.15.43:0/964054933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f816810f280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8164006f00 tx=0x7f8164008040 comp rx=0 tx=0).stop 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 shutdown_connections 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 --2- 172.21.15.43:0/964054933 >> v2:172.21.15.43:6800/1424089977 conn(0x7f81540394c0 0x7f815403b970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 --2- 172.21.15.43:0/964054933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81681005b0 0x7f816810f280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 >> 172.21.15.43:0/964054933 conn(0x7f81680fbe10 msgr2=0x7f81680fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 shutdown_connections 2024-09-11T08:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:45.134+0000 7f816f7bb640 1 -- 172.21.15.43:0/964054933 wait complete. 2024-09-11T08:25:45.657 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:25:45.657 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-11T08:25:45.666 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:25:45.666 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T08:25:45.666 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-11T08:25:45.666 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T08:25:45.668 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-09-11T08:25:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043.front.sepia.ceph.com:8443/ 2024-09-11T08:25:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-09-11T08:25:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: Password: 1s0nyu3pc8 2024-09-11T08:25:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:45.669 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config directory 2024-09-11T08:25:45.669 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling autotune for osd_memory_target 2024-09-11T08:25:46.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:46 smithi043 ceph-mon[29355]: mgrmap e13: y(active, since 6s) 2024-09-11T08:25:46.821 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.093+0000 7f93fb0f2640 1 Processor -- start 2024-09-11T08:25:46.821 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.093+0000 7f93fb0f2640 1 -- start start 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.093+0000 7f93fb0f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4075b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.093+0000 7f93fb0f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4077850 con 0x7f93f4077440 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.093+0000 7f93f8e8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.093+0000 7f93f8e8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37908/0 (socket says 172.21.15.43:37908) 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.093+0000 7f93f8e8e640 1 -- 172.21.15.43:0/2701137330 learned_addr learned my addr 172.21.15.43:0/2701137330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.094+0000 7f93f8e8e640 1 -- 172.21.15.43:0/2701137330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4077990 con 0x7f93f4077440 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.094+0000 7f93f8e8e640 1 --2- 172.21.15.43:0/2701137330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4075b00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009840 tx=0x7f93e80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa5cd143de0909b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93f37fe640 1 -- 172.21.15.43:0/2701137330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93e802ae60 con 0x7f93f4077440 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93f37fe640 1 -- 172.21.15.43:0/2701137330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f93e80328b0 con 0x7f93f4077440 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93f37fe640 1 -- 172.21.15.43:0/2701137330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93e8031740 con 0x7f93f4077440 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93fb0f2640 1 -- 172.21.15.43:0/2701137330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 msgr2=0x7f93f4075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93fb0f2640 1 --2- 172.21.15.43:0/2701137330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4075b00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009840 tx=0x7f93e80091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93fb0f2640 1 -- 172.21.15.43:0/2701137330 shutdown_connections 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93fb0f2640 1 --2- 172.21.15.43:0/2701137330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4075b00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93fb0f2640 1 -- 172.21.15.43:0/2701137330 >> 172.21.15.43:0/2701137330 conn(0x7f93f40fbe10 msgr2=0x7f93f40fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93fb0f2640 1 -- 172.21.15.43:0/2701137330 shutdown_connections 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.095+0000 7f93fb0f2640 1 -- 172.21.15.43:0/2701137330 wait complete. 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.096+0000 7f93fb0f2640 1 Processor -- start 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.096+0000 7f93fb0f2640 1 -- start start 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.096+0000 7f93fb0f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4111430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.096+0000 7f93fb0f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4111970 con 0x7f93f4077440 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.096+0000 7f93f8e8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4111430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.096+0000 7f93f8e8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4111430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37924/0 (socket says 172.21.15.43:37924) 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.096+0000 7f93f8e8e640 1 -- 172.21.15.43:0/3088074271 learned_addr learned my addr 172.21.15.43:0/3088074271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:46.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.097+0000 7f93f8e8e640 1 -- 172.21.15.43:0/3088074271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e8009450 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.097+0000 7f93f8e8e640 1 --2- 172.21.15.43:0/3088074271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4111430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93e8031450 tx=0x7f93e802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.097+0000 7f93f17fa640 1 -- 172.21.15.43:0/3088074271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93e8031bc0 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.097+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f4111b70 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.097+0000 7f93f17fa640 1 -- 172.21.15.43:0/3088074271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f93e8031d20 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.097+0000 7f93f17fa640 1 -- 172.21.15.43:0/3088074271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93e803b420 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.097+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f4111fe0 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.098+0000 7f93f17fa640 1 -- 172.21.15.43:0/3088074271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f93e8039070 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.098+0000 7f93f17fa640 1 --2- 172.21.15.43:0/3088074271 >> v2:172.21.15.43:6800/1424089977 conn(0x7f93e4039510 0x7f93e403b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.098+0000 7f93f17fa640 1 -- 172.21.15.43:0/3088074271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f93e806df00 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.098+0000 7f93f3fff640 1 --2- 172.21.15.43:0/3088074271 >> v2:172.21.15.43:6800/1424089977 conn(0x7f93e4039510 0x7f93e403b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.098+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f405ed70 con 0x7f93f4077440 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.099+0000 7f93f3fff640 1 --2- 172.21.15.43:0/3088074271 >> v2:172.21.15.43:6800/1424089977 conn(0x7f93e4039510 0x7f93e403b9c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006fd0 tx=0x7f93e0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:46.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.103+0000 7f93f17fa640 1 -- 172.21.15.43:0/3088074271 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f93e8041020 con 0x7f93f4077440 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.266+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f93f404b720 con 0x7f93f4077440 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.267+0000 7f93f17fa640 1 -- 172.21.15.43:0/3088074271 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f93e80550c0 con 0x7f93f4077440 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 >> v2:172.21.15.43:6800/1424089977 conn(0x7f93e4039510 msgr2=0x7f93e403b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 --2- 172.21.15.43:0/3088074271 >> v2:172.21.15.43:6800/1424089977 conn(0x7f93e4039510 0x7f93e403b9c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006fd0 tx=0x7f93e0006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 msgr2=0x7f93f4111430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 --2- 172.21.15.43:0/3088074271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4111430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93e8031450 tx=0x7f93e802c040 comp rx=0 tx=0).stop 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 shutdown_connections 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 --2- 172.21.15.43:0/3088074271 >> v2:172.21.15.43:6800/1424089977 conn(0x7f93e4039510 0x7f93e403b9c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 --2- 172.21.15.43:0/3088074271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f4077440 0x7f93f4111430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 >> 172.21.15.43:0/3088074271 conn(0x7f93f40fbe10 msgr2=0x7f93f40fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 shutdown_connections 2024-09-11T08:25:46.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:46.269+0000 7f93fb0f2640 1 -- 172.21.15.43:0/3088074271 wait complete. 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.259+0000 7f339c7bf640 1 Processor -- start 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.260+0000 7f339c7bf640 1 -- start start 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.260+0000 7f339c7bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33940ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.260+0000 7f339c7bf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33940ffdf0 con 0x7f33940ff4a0 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.260+0000 7f339a55b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33940ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.260+0000 7f339a55b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33940ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37940/0 (socket says 172.21.15.43:37940) 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.260+0000 7f339a55b640 1 -- 172.21.15.43:0/4079466954 learned_addr learned my addr 172.21.15.43:0/4079466954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.260+0000 7f339a55b640 1 -- 172.21.15.43:0/4079466954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940fff30 con 0x7f33940ff4a0 2024-09-11T08:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.261+0000 7f339a55b640 1 --2- 172.21.15.43:0/4079466954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33940ff8b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3390009840 tx=0x7f33900091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac87474118cd8e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.261+0000 7f3399559640 1 -- 172.21.15.43:0/4079466954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f339002add0 con 0x7f33940ff4a0 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.261+0000 7f3399559640 1 -- 172.21.15.43:0/4079466954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f33900328b0 con 0x7f33940ff4a0 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.261+0000 7f3399559640 1 -- 172.21.15.43:0/4079466954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3390031690 con 0x7f33940ff4a0 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.262+0000 7f339c7bf640 1 -- 172.21.15.43:0/4079466954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 msgr2=0x7f33940ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.262+0000 7f339c7bf640 1 --2- 172.21.15.43:0/4079466954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33940ff8b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3390009840 tx=0x7f33900091d0 comp rx=0 tx=0).stop 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.262+0000 7f339c7bf640 1 -- 172.21.15.43:0/4079466954 shutdown_connections 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.262+0000 7f339c7bf640 1 --2- 172.21.15.43:0/4079466954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33940ff8b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.262+0000 7f339c7bf640 1 -- 172.21.15.43:0/4079466954 >> 172.21.15.43:0/4079466954 conn(0x7f33940faca0 msgr2=0x7f33940fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.262+0000 7f339c7bf640 1 -- 172.21.15.43:0/4079466954 shutdown_connections 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.262+0000 7f339c7bf640 1 -- 172.21.15.43:0/4079466954 wait complete. 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339c7bf640 1 Processor -- start 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339c7bf640 1 -- start start 2024-09-11T08:25:48.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339c7bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33941056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339c7bf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394105c10 con 0x7f33940ff4a0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339a55b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33941056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339a55b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33941056d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37944/0 (socket says 172.21.15.43:37944) 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339a55b640 1 -- 172.21.15.43:0/3971775851 learned_addr learned my addr 172.21.15.43:0/3971775851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339a55b640 1 -- 172.21.15.43:0/3971775851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009450 con 0x7f33940ff4a0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339a55b640 1 --2- 172.21.15.43:0/3971775851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33941056d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3390031450 tx=0x7f3390008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f338affd640 1 -- 172.21.15.43:0/3971775851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3390031ca0 con 0x7f33940ff4a0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f338affd640 1 -- 172.21.15.43:0/3971775851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f3390031e00 con 0x7f33940ff4a0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.263+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3394103ca0 con 0x7f33940ff4a0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.264+0000 7f338affd640 1 -- 172.21.15.43:0/3971775851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f339003a540 con 0x7f33940ff4a0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.264+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3394104110 con 0x7f33940ff4a0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.265+0000 7f338affd640 1 -- 172.21.15.43:0/3971775851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f3390039070 con 0x7f33940ff4a0 2024-09-11T08:25:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.265+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339404b720 con 0x7f33940ff4a0 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.265+0000 7f338affd640 1 --2- 172.21.15.43:0/3971775851 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3380039560 0x7f338003ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.265+0000 7f338affd640 1 -- 172.21.15.43:0/3971775851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f339006cb70 con 0x7f33940ff4a0 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.265+0000 7f3399d5a640 1 --2- 172.21.15.43:0/3971775851 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3380039560 0x7f338003ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.265+0000 7f3399d5a640 1 --2- 172.21.15.43:0/3971775851 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3380039560 0x7f338003ba10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3384006fd0 tx=0x7f3384006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.269+0000 7f338affd640 1 -- 172.21.15.43:0/3971775851 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3390044400 con 0x7f33940ff4a0 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.519+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f3394104a60 con 0x7f33940ff4a0 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.521+0000 7f338affd640 1 -- 172.21.15.43:0/3971775851 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f339003f070 con 0x7f33940ff4a0 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3380039560 msgr2=0x7f338003ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 --2- 172.21.15.43:0/3971775851 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3380039560 0x7f338003ba10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3384006fd0 tx=0x7f3384006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 msgr2=0x7f33941056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 --2- 172.21.15.43:0/3971775851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33941056d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3390031450 tx=0x7f3390008040 comp rx=0 tx=0).stop 2024-09-11T08:25:48.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 shutdown_connections 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 --2- 172.21.15.43:0/3971775851 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3380039560 0x7f338003ba10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 --2- 172.21.15.43:0/3971775851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33940ff4a0 0x7f33941056d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 >> 172.21.15.43:0/3971775851 conn(0x7f33940faca0 msgr2=0x7f33940fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 shutdown_connections 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T08:25:47.523+0000 7f339c7bf640 1 -- 172.21.15.43:0/3971775851 wait complete. 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /sbin/cephadm shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /sbin/cephadm shell 2024-09-11T08:25:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:25:48.182 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-09-11T08:25:48.218 INFO:tasks.cephadm:Fetching config... 2024-09-11T08:25:48.219 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:25:48.219 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T08:25:48.238 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T08:25:48.238 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:25:48.238 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T08:25:48.296 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T08:25:48.296 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:25:48.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T08:25:48.363 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T08:25:48.364 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:25:48.364 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T08:25:48.420 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T08:25:48.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC52znb6C5Dyya2KXz/qcQiTztSIK7rrdWaIDAqiKqNukZ4xG6UPMGKq6TPON4YGQJNr9D2/T3ukTsLyZvDtU5FQ851DILdgDFC+hD6E/R/cGQqAt/WNiHd5VIDACu7jQJjnOtQAqYSCkcPGlI8nCh9MW/Nc465XweufMRRvZnmwvYbSYiu/3bfX9Czof5RnGmlINSE9rdqNwy9Jt5YwKdfMFYGtToEYzRBguLaVGThw4sGnOKeNCRZQ8wmwqmuS6pnaMcrveOb9wgB8HbwX8RoadGaRtFogU0YV9l7aUKb57ZjJwzoJCZIyrQ8CR+eyIJXmHTWD48AqFFl8MYYa6GDZHp7UzOvrWOdH9ld9HWLm0aXgtMeD853LgQwcm5SAZTAKLNBk5xHm4ALKnkXp/MXOaaqBv+K+W3CN4WnF0DtkoYXmdwfK73/UpiMyelBZO4WFIHgoHXLqq/Fy/GOvzA6OYnHgLYqSLGBLwncP5Z5H7ETQmBX0kLu7yMWXOIMCts= ceph-327ee848-7017-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:25:48.521 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC52znb6C5Dyya2KXz/qcQiTztSIK7rrdWaIDAqiKqNukZ4xG6UPMGKq6TPON4YGQJNr9D2/T3ukTsLyZvDtU5FQ851DILdgDFC+hD6E/R/cGQqAt/WNiHd5VIDACu7jQJjnOtQAqYSCkcPGlI8nCh9MW/Nc465XweufMRRvZnmwvYbSYiu/3bfX9Czof5RnGmlINSE9rdqNwy9Jt5YwKdfMFYGtToEYzRBguLaVGThw4sGnOKeNCRZQ8wmwqmuS6pnaMcrveOb9wgB8HbwX8RoadGaRtFogU0YV9l7aUKb57ZjJwzoJCZIyrQ8CR+eyIJXmHTWD48AqFFl8MYYa6GDZHp7UzOvrWOdH9ld9HWLm0aXgtMeD853LgQwcm5SAZTAKLNBk5xHm4ALKnkXp/MXOaaqBv+K+W3CN4WnF0DtkoYXmdwfK73/UpiMyelBZO4WFIHgoHXLqq/Fy/GOvzA6OYnHgLYqSLGBLwncP5Z5H7ETQmBX0kLu7yMWXOIMCts= ceph-327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:25:48.535 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC52znb6C5Dyya2KXz/qcQiTztSIK7rrdWaIDAqiKqNukZ4xG6UPMGKq6TPON4YGQJNr9D2/T3ukTsLyZvDtU5FQ851DILdgDFC+hD6E/R/cGQqAt/WNiHd5VIDACu7jQJjnOtQAqYSCkcPGlI8nCh9MW/Nc465XweufMRRvZnmwvYbSYiu/3bfX9Czof5RnGmlINSE9rdqNwy9Jt5YwKdfMFYGtToEYzRBguLaVGThw4sGnOKeNCRZQ8wmwqmuS6pnaMcrveOb9wgB8HbwX8RoadGaRtFogU0YV9l7aUKb57ZjJwzoJCZIyrQ8CR+eyIJXmHTWD48AqFFl8MYYa6GDZHp7UzOvrWOdH9ld9HWLm0aXgtMeD853LgQwcm5SAZTAKLNBk5xHm4ALKnkXp/MXOaaqBv+K+W3CN4WnF0DtkoYXmdwfK73/UpiMyelBZO4WFIHgoHXLqq/Fy/GOvzA6OYnHgLYqSLGBLwncP5Z5H7ETQmBX0kLu7yMWXOIMCts= ceph-327ee848-7017-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:25:48.662 INFO:teuthology.orchestra.run.smithi172.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC52znb6C5Dyya2KXz/qcQiTztSIK7rrdWaIDAqiKqNukZ4xG6UPMGKq6TPON4YGQJNr9D2/T3ukTsLyZvDtU5FQ851DILdgDFC+hD6E/R/cGQqAt/WNiHd5VIDACu7jQJjnOtQAqYSCkcPGlI8nCh9MW/Nc465XweufMRRvZnmwvYbSYiu/3bfX9Czof5RnGmlINSE9rdqNwy9Jt5YwKdfMFYGtToEYzRBguLaVGThw4sGnOKeNCRZQ8wmwqmuS6pnaMcrveOb9wgB8HbwX8RoadGaRtFogU0YV9l7aUKb57ZjJwzoJCZIyrQ8CR+eyIJXmHTWD48AqFFl8MYYa6GDZHp7UzOvrWOdH9ld9HWLm0aXgtMeD853LgQwcm5SAZTAKLNBk5xHm4ALKnkXp/MXOaaqBv+K+W3CN4WnF0DtkoYXmdwfK73/UpiMyelBZO4WFIHgoHXLqq/Fy/GOvzA6OYnHgLYqSLGBLwncP5Z5H7ETQmBX0kLu7yMWXOIMCts= ceph-327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:25:48.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T08:25:48.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3971775851' entity='client.admin' 2024-09-11T08:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:25:51.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:51.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:51.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:25:51.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:51.075 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:51.075 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.260+0000 7f88c24eb640 1 -- 172.21.15.43:0/3009576976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 msgr2=0x7f88bc0fb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.260+0000 7f88c24eb640 1 --2- 172.21.15.43:0/3009576976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 0x7f88bc0fb6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88ac004850 tx=0x7f88ac009da0 comp rx=0 tx=0).stop 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.260+0000 7f88c24eb640 1 -- 172.21.15.43:0/3009576976 shutdown_connections 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.260+0000 7f88c24eb640 1 --2- 172.21.15.43:0/3009576976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 0x7f88bc0fb6f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.260+0000 7f88c24eb640 1 -- 172.21.15.43:0/3009576976 >> 172.21.15.43:0/3009576976 conn(0x7f88bc0f6ac0 msgr2=0x7f88bc0f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.260+0000 7f88c24eb640 1 -- 172.21.15.43:0/3009576976 shutdown_connections 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.260+0000 7f88c24eb640 1 -- 172.21.15.43:0/3009576976 wait complete. 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.261+0000 7f88c24eb640 1 Processor -- start 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.261+0000 7f88c24eb640 1 -- start start 2024-09-11T08:25:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.261+0000 7f88c24eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 0x7f88bc10be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.261+0000 7f88c24eb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc10c3b0 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.261+0000 7f88bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 0x7f88bc10be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.261+0000 7f88bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 0x7f88bc10be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37958/0 (socket says 172.21.15.43:37958) 2024-09-11T08:25:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.261+0000 7f88bbfff640 1 -- 172.21.15.43:0/236272891 learned_addr learned my addr 172.21.15.43:0/236272891 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.262+0000 7f88bbfff640 1 -- 172.21.15.43:0/236272891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac005040 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.262+0000 7f88bbfff640 1 --2- 172.21.15.43:0/236272891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 0x7f88bc10be70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88ac004820 tx=0x7f88ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:51.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.262+0000 7f88b8ff9640 1 -- 172.21.15.43:0/236272891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88ac038070 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.262+0000 7f88b8ff9640 1 -- 172.21.15.43:0/236272891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f88ac0098a0 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.262+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc10c5b0 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.262+0000 7f88b8ff9640 1 -- 172.21.15.43:0/236272891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88ac03c5b0 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.262+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc10c9c0 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.263+0000 7f88b8ff9640 1 -- 172.21.15.43:0/236272891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f88ac030020 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.263+0000 7f88b8ff9640 1 --2- 172.21.15.43:0/236272891 >> v2:172.21.15.43:6800/1424089977 conn(0x7f88a4039510 0x7f88a403b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.263+0000 7f88b8ff9640 1 -- 172.21.15.43:0/236272891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f88ac06be00 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.263+0000 7f88bb7fe640 1 --2- 172.21.15.43:0/236272891 >> v2:172.21.15.43:6800/1424089977 conn(0x7f88a4039510 0x7f88a403b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.263+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc05ed70 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.264+0000 7f88bb7fe640 1 --2- 172.21.15.43:0/236272891 >> v2:172.21.15.43:6800/1424089977 conn(0x7f88a4039510 0x7f88a403b9c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f88b0006fd0 tx=0x7f88b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:51.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.268+0000 7f88b8ff9640 1 -- 172.21.15.43:0/236272891 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f88ac0453f0 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.439+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f88bc04b720 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.443+0000 7f88b8ff9640 1 -- 172.21.15.43:0/236272891 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f88ac043b00 con 0x7f88bc0fb2e0 2024-09-11T08:25:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.446+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 >> v2:172.21.15.43:6800/1424089977 conn(0x7f88a4039510 msgr2=0x7f88a403b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.446+0000 7f88c24eb640 1 --2- 172.21.15.43:0/236272891 >> v2:172.21.15.43:6800/1424089977 conn(0x7f88a4039510 0x7f88a403b9c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f88b0006fd0 tx=0x7f88b0006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.446+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 msgr2=0x7f88bc10be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.446+0000 7f88c24eb640 1 --2- 172.21.15.43:0/236272891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 0x7f88bc10be70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88ac004820 tx=0x7f88ac009da0 comp rx=0 tx=0).stop 2024-09-11T08:25:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.447+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 shutdown_connections 2024-09-11T08:25:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.447+0000 7f88c24eb640 1 --2- 172.21.15.43:0/236272891 >> v2:172.21.15.43:6800/1424089977 conn(0x7f88a4039510 0x7f88a403b9c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.447+0000 7f88c24eb640 1 --2- 172.21.15.43:0/236272891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88bc0fb2e0 0x7f88bc10be70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.447+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 >> 172.21.15.43:0/236272891 conn(0x7f88bc0f6ac0 msgr2=0x7f88bc064990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.447+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 shutdown_connections 2024-09-11T08:25:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:51.447+0000 7f88c24eb640 1 -- 172.21.15.43:0/236272891 wait complete. 2024-09-11T08:25:52.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/236272891' entity='client.admin' 2024-09-11T08:25:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:52 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:52 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:52.726 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T08:25:52.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T08:25:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 -- 172.21.15.43:0/21920907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 msgr2=0x7f14480fb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 --2- 172.21.15.43:0/21920907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 0x7f14480fb6f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1440004850 tx=0x7f1440009da0 comp rx=0 tx=0).stop 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 -- 172.21.15.43:0/21920907 shutdown_connections 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 --2- 172.21.15.43:0/21920907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 0x7f14480fb6f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 -- 172.21.15.43:0/21920907 >> 172.21.15.43:0/21920907 conn(0x7f14480f6ac0 msgr2=0x7f14480f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 -- 172.21.15.43:0/21920907 shutdown_connections 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 -- 172.21.15.43:0/21920907 wait complete. 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 Processor -- start 2024-09-11T08:25:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.557+0000 7f144cd3b640 1 -- start start 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f144cd3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 0x7f144810e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f144cd3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144810e5e0 con 0x7f14480fb2e0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f144659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 0x7f144810e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f144659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 0x7f144810e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47226/0 (socket says 172.21.15.43:47226) 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f144659c640 1 -- 172.21.15.43:0/3457113293 learned_addr learned my addr 172.21.15.43:0/3457113293 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f144659c640 1 -- 172.21.15.43:0/3457113293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440005040 con 0x7f14480fb2e0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f144659c640 1 --2- 172.21.15.43:0/3457113293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 0x7f144810e0a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1440004820 tx=0x7f1440009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f143effd640 1 -- 172.21.15.43:0/3457113293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1440038070 con 0x7f14480fb2e0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f143effd640 1 -- 172.21.15.43:0/3457113293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f14400098a0 con 0x7f14480fb2e0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144810e7e0 con 0x7f14480fb2e0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.558+0000 7f143effd640 1 -- 172.21.15.43:0/3457113293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f144003c5b0 con 0x7f14480fb2e0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.559+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144810ebf0 con 0x7f14480fb2e0 2024-09-11T08:25:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.559+0000 7f143effd640 1 -- 172.21.15.43:0/3457113293 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f1440030020 con 0x7f14480fb2e0 2024-09-11T08:25:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.560+0000 7f143effd640 1 --2- 172.21.15.43:0/3457113293 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1434039560 0x7f143403ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.560+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144805ed70 con 0x7f14480fb2e0 2024-09-11T08:25:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.560+0000 7f143effd640 1 -- 172.21.15.43:0/3457113293 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f144006bd80 con 0x7f14480fb2e0 2024-09-11T08:25:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.560+0000 7f1445d9b640 1 --2- 172.21.15.43:0/3457113293 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1434039560 0x7f143403ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.560+0000 7f1445d9b640 1 --2- 172.21.15.43:0/3457113293 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1434039560 0x7f143403ba10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1430006fd0 tx=0x7f1430006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.564+0000 7f143effd640 1 -- 172.21.15.43:0/3457113293 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f1440071020 con 0x7f14480fb2e0 2024-09-11T08:25:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.733+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f14481004f0 con 0x7f1434039560 2024-09-11T08:25:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.736+0000 7f143effd640 1 -- 172.21.15.43:0/3457113293 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f14481004f0 con 0x7f1434039560 2024-09-11T08:25:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1434039560 msgr2=0x7f143403ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 --2- 172.21.15.43:0/3457113293 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1434039560 0x7f143403ba10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1430006fd0 tx=0x7f1430006db0 comp rx=0 tx=0).stop 2024-09-11T08:25:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 msgr2=0x7f144810e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 --2- 172.21.15.43:0/3457113293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 0x7f144810e0a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1440004820 tx=0x7f1440009da0 comp rx=0 tx=0).stop 2024-09-11T08:25:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 shutdown_connections 2024-09-11T08:25:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 --2- 172.21.15.43:0/3457113293 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1434039560 0x7f143403ba10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 --2- 172.21.15.43:0/3457113293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14480fb2e0 0x7f144810e0a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 >> 172.21.15.43:0/3457113293 conn(0x7f14480f6ac0 msgr2=0x7f14480f8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 shutdown_connections 2024-09-11T08:25:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:54.738+0000 7f144cd3b640 1 -- 172.21.15.43:0/3457113293 wait complete. 2024-09-11T08:25:55.096 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:54 smithi043 ceph-mon[29355]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:55.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:54 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:55.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:54 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:25:55.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:54 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:25:55.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:54 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:25:55.466 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi172 2024-09-11T08:25:55.466 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:25:55.467 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.conf 2024-09-11T08:25:55.487 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:25:55.487 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:25:55.545 INFO:tasks.cephadm:Adding host smithi172 to orchestrator... 2024-09-11T08:25:55.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch host add smithi172 2024-09-11T08:25:55.799 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:25:56.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:56 smithi043 ceph-mon[29355]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-11T08:25:56.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:56 smithi043 ceph-mon[29355]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:25:56.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:56 smithi043 ceph-mon[29355]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:25:56.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:56 smithi043 ceph-mon[29355]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:25:56.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:56.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:56.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:25:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.283+0000 7f7f71494640 1 -- 172.21.15.43:0/4262846133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 msgr2=0x7f7f6c0fd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:25:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.283+0000 7f7f71494640 1 --2- 172.21.15.43:0/4262846133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 0x7f7f6c0fd5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c004850 tx=0x7f7f5c009da0 comp rx=0 tx=0).stop 2024-09-11T08:25:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.284+0000 7f7f71494640 1 -- 172.21.15.43:0/4262846133 shutdown_connections 2024-09-11T08:25:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.284+0000 7f7f71494640 1 --2- 172.21.15.43:0/4262846133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 0x7f7f6c0fd5f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:25:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.284+0000 7f7f71494640 1 -- 172.21.15.43:0/4262846133 >> 172.21.15.43:0/4262846133 conn(0x7f7f6c0f6cd0 msgr2=0x7f7f6c0f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:25:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.284+0000 7f7f71494640 1 -- 172.21.15.43:0/4262846133 shutdown_connections 2024-09-11T08:25:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.284+0000 7f7f71494640 1 -- 172.21.15.43:0/4262846133 wait complete. 2024-09-11T08:25:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.284+0000 7f7f71494640 1 Processor -- start 2024-09-11T08:25:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.284+0000 7f7f71494640 1 -- start start 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f71494640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 0x7f7f6c111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f71494640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c112230 con 0x7f7f6c0fd220 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 0x7f7f6c111cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 0x7f7f6c111cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47240/0 (socket says 172.21.15.43:47240) 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f6affd640 1 -- 172.21.15.43:0/3229985509 learned_addr learned my addr 172.21.15.43:0/3229985509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f6affd640 1 -- 172.21.15.43:0/3229985509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c005040 con 0x7f7f6c0fd220 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f6affd640 1 --2- 172.21.15.43:0/3229985509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 0x7f7f6c111cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c004820 tx=0x7f7f5c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f637fe640 1 -- 172.21.15.43:0/3229985509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c038070 con 0x7f7f6c0fd220 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f637fe640 1 -- 172.21.15.43:0/3229985509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f7f5c009430 con 0x7f7f6c0fd220 2024-09-11T08:25:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c112430 con 0x7f7f6c0fd220 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f637fe640 1 -- 172.21.15.43:0/3229985509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c0335c0 con 0x7f7f6c0fd220 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.285+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c112820 con 0x7f7f6c0fd220 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.286+0000 7f7f637fe640 1 -- 172.21.15.43:0/3229985509 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f7f5c030020 con 0x7f7f6c0fd220 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.286+0000 7f7f637fe640 1 --2- 172.21.15.43:0/3229985509 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7f580394d0 0x7f7f5803b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.286+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c065dc0 con 0x7f7f6c0fd220 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.287+0000 7f7f637fe640 1 -- 172.21.15.43:0/3229985509 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f7f5c06c580 con 0x7f7f6c0fd220 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.287+0000 7f7f63fff640 1 --2- 172.21.15.43:0/3229985509 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7f580394d0 0x7f7f5803b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.287+0000 7f7f63fff640 1 --2- 172.21.15.43:0/3229985509 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7f580394d0 0x7f7f5803b980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f54006fd0 tx=0x7f7f54006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:25:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.291+0000 7f7f637fe640 1 -- 172.21.15.43:0/3229985509 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f7f5c047440 con 0x7f7f6c0fd220 2024-09-11T08:25:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:25:57.462+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f6c105aa0 con 0x7f7f580394d0 2024-09-11T08:25:58.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:57 smithi043 ceph-mon[29355]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:25:59.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:25:58 smithi043 ceph-mon[29355]: Deploying cephadm binary to smithi172 2024-09-11T08:26:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:00 smithi043 ceph-mon[29355]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.761+0000 7f7f637fe640 1 -- 172.21.15.43:0/3229985509 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7f6c105aa0 con 0x7f7f580394d0 2024-09-11T08:26:00.765 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi172' with addr '172.21.15.172' 2024-09-11T08:26:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.763+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7f580394d0 msgr2=0x7f7f5803b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.763+0000 7f7f71494640 1 --2- 172.21.15.43:0/3229985509 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7f580394d0 0x7f7f5803b980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f54006fd0 tx=0x7f7f54006db0 comp rx=0 tx=0).stop 2024-09-11T08:26:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.763+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 msgr2=0x7f7f6c111cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.763+0000 7f7f71494640 1 --2- 172.21.15.43:0/3229985509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 0x7f7f6c111cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c004820 tx=0x7f7f5c009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.763+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 shutdown_connections 2024-09-11T08:26:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.763+0000 7f7f71494640 1 --2- 172.21.15.43:0/3229985509 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7f580394d0 0x7f7f5803b980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.763+0000 7f7f71494640 1 --2- 172.21.15.43:0/3229985509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0fd220 0x7f7f6c111cf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.763+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 >> 172.21.15.43:0/3229985509 conn(0x7f7f6c0f6cd0 msgr2=0x7f7f6c102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.764+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 shutdown_connections 2024-09-11T08:26:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:00.764+0000 7f7f71494640 1 -- 172.21.15.43:0/3229985509 wait complete. 2024-09-11T08:26:01.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T08:26:01.585 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:26:01.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:01.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:01 smithi043 ceph-mon[29355]: Added host smithi172 2024-09-11T08:26:01.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:01.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:01.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:01 smithi043 ceph-mon[29355]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:01.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:03.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1361724494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 msgr2=0x7fb278102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:03.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 --2- 172.21.15.43:0/1361724494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 0x7fb278102540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb268004850 tx=0x7fb268009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:03.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1361724494 shutdown_connections 2024-09-11T08:26:03.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 --2- 172.21.15.43:0/1361724494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 0x7fb278102540 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1361724494 >> 172.21.15.43:0/1361724494 conn(0x7fb278071040 msgr2=0x7fb278071440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1361724494 shutdown_connections 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1361724494 wait complete. 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 Processor -- start 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.195+0000 7fb27e1f5640 1 -- start start 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb27e1f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 0x7fb278111c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb27e1f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278112160 con 0x7fb278102170 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb2777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 0x7fb278111c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb2777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 0x7fb278111c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44118/0 (socket says 172.21.15.43:44118) 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb2777fe640 1 -- 172.21.15.43:0/1638363874 learned_addr learned my addr 172.21.15.43:0/1638363874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb2777fe640 1 -- 172.21.15.43:0/1638363874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb268005040 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb2777fe640 1 --2- 172.21.15.43:0/1638363874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 0x7fb278111c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb268003080 tx=0x7fb268009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb26ffff640 1 -- 172.21.15.43:0/1638363874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb268038070 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb26ffff640 1 -- 172.21.15.43:0/1638363874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fb2680093b0 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb278112360 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb26ffff640 1 -- 172.21.15.43:0/1638363874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb26803c410 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.196+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2781127d0 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.197+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2780fd8e0 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.197+0000 7fb26ffff640 1 -- 172.21.15.43:0/1638363874 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7fb268030020 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.198+0000 7fb26ffff640 1 --2- 172.21.15.43:0/1638363874 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb264039560 0x7fb26403ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.198+0000 7fb26ffff640 1 -- 172.21.15.43:0/1638363874 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fb26806bc50 con 0x7fb278102170 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.198+0000 7fb276ffd640 1 --2- 172.21.15.43:0/1638363874 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb264039560 0x7fb26403ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.198+0000 7fb276ffd640 1 --2- 172.21.15.43:0/1638363874 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb264039560 0x7fb26403ba10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb260006fd0 tx=0x7fb260006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:03.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.202+0000 7fb26ffff640 1 -- 172.21.15.43:0/1638363874 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb26803b7e0 con 0x7fb278102170 2024-09-11T08:26:03.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.371+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb27810d330 con 0x7fb264039560 2024-09-11T08:26:03.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:03 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.373+0000 7fb26ffff640 1 -- 172.21.15.43:0/1638363874 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fb27810d330 con 0x7fb264039560 2024-09-11T08:26:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:26:03.373 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}] 2024-09-11T08:26:03.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.374+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb264039560 msgr2=0x7fb26403ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:03.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.374+0000 7fb27e1f5640 1 --2- 172.21.15.43:0/1638363874 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb264039560 0x7fb26403ba10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb260006fd0 tx=0x7fb260006db0 comp rx=0 tx=0).stop 2024-09-11T08:26:03.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.374+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 msgr2=0x7fb278111c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:03.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.374+0000 7fb27e1f5640 1 --2- 172.21.15.43:0/1638363874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 0x7fb278111c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb268003080 tx=0x7fb268009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:03.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.375+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 shutdown_connections 2024-09-11T08:26:03.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.375+0000 7fb27e1f5640 1 --2- 172.21.15.43:0/1638363874 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb264039560 0x7fb26403ba10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:03.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.375+0000 7fb27e1f5640 1 --2- 172.21.15.43:0/1638363874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb278102170 0x7fb278111c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:03.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.375+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 >> 172.21.15.43:0/1638363874 conn(0x7fb278071040 msgr2=0x7fb278103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:03.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.375+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 shutdown_connections 2024-09-11T08:26:03.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:03.375+0000 7fb27e1f5640 1 -- 172.21.15.43:0/1638363874 wait complete. 2024-09-11T08:26:03.993 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T08:26:03.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T08:26:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:26:04.301 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:04 smithi043 ceph-mon[29355]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:04.302 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:04 smithi043 ceph-mon[29355]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:26:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 -- 172.21.15.43:0/4289258399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 msgr2=0x7f13481014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 --2- 172.21.15.43:0/4289258399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 0x7f13481014d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1338004850 tx=0x7f1338009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 -- 172.21.15.43:0/4289258399 shutdown_connections 2024-09-11T08:26:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 --2- 172.21.15.43:0/4289258399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 0x7f13481014d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 -- 172.21.15.43:0/4289258399 >> 172.21.15.43:0/4289258399 conn(0x7f13480f6cd0 msgr2=0x7f13480f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 -- 172.21.15.43:0/4289258399 shutdown_connections 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 -- 172.21.15.43:0/4289258399 wait complete. 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 Processor -- start 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.844+0000 7f134e6f1640 1 -- start start 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f134e6f1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 0x7f13480f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f134e6f1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480f76b0 con 0x7f1348101100 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f1347fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 0x7f13480f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f1347fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 0x7f13480f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44136/0 (socket says 172.21.15.43:44136) 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f1347fff640 1 -- 172.21.15.43:0/2668224476 learned_addr learned my addr 172.21.15.43:0/2668224476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f1347fff640 1 -- 172.21.15.43:0/2668224476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1338005040 con 0x7f1348101100 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f1347fff640 1 --2- 172.21.15.43:0/2668224476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 0x7f13480f7170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1338003080 tx=0x7f1338009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f1344ff9640 1 -- 172.21.15.43:0/2668224476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1338038070 con 0x7f1348101100 2024-09-11T08:26:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f1344ff9640 1 -- 172.21.15.43:0/2668224476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f13380093b0 con 0x7f1348101100 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13480f78b0 con 0x7f1348101100 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.845+0000 7f1344ff9640 1 -- 172.21.15.43:0/2668224476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f133803c410 con 0x7f1348101100 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.846+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13480f7ca0 con 0x7f1348101100 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.846+0000 7f1344ff9640 1 -- 172.21.15.43:0/2668224476 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f1338030020 con 0x7f1348101100 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.847+0000 7f1344ff9640 1 --2- 172.21.15.43:0/2668224476 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1330039560 0x7f133003ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.847+0000 7f1344ff9640 1 -- 172.21.15.43:0/2668224476 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f133806bc30 con 0x7f1348101100 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.847+0000 7f13477fe640 1 --2- 172.21.15.43:0/2668224476 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1330039560 0x7f133003ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.847+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13480fa330 con 0x7f1348101100 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.847+0000 7f13477fe640 1 --2- 172.21.15.43:0/2668224476 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1330039560 0x7f133003ba10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f133c006fd0 tx=0x7f133c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:05.852+0000 7f1344ff9640 1 -- 172.21.15.43:0/2668224476 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f1338041080 con 0x7f1348101100 2024-09-11T08:26:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.022+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f13480fc8a0 con 0x7f1348101100 2024-09-11T08:26:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:06.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.939+0000 7f1344ff9640 1 -- 172.21.15.43:0/2668224476 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f133803aaf0 con 0x7f1348101100 2024-09-11T08:26:06.940 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-09-11T08:26:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1330039560 msgr2=0x7f133003ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 --2- 172.21.15.43:0/2668224476 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1330039560 0x7f133003ba10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f133c006fd0 tx=0x7f133c006db0 comp rx=0 tx=0).stop 2024-09-11T08:26:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 msgr2=0x7f13480f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 --2- 172.21.15.43:0/2668224476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 0x7f13480f7170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1338003080 tx=0x7f1338009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 shutdown_connections 2024-09-11T08:26:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 --2- 172.21.15.43:0/2668224476 >> v2:172.21.15.43:6800/1424089977 conn(0x7f1330039560 0x7f133003ba10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 --2- 172.21.15.43:0/2668224476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348101100 0x7f13480f7170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 >> 172.21.15.43:0/2668224476 conn(0x7f13480f6cd0 msgr2=0x7f1348065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 shutdown_connections 2024-09-11T08:26:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:06.942+0000 7f134e6f1640 1 -- 172.21.15.43:0/2668224476 wait complete. 2024-09-11T08:26:07.297 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2668224476' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T08:26:07.578 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-09-11T08:26:07.579 INFO:tasks.cephadm:Adding mon.c on smithi043 2024-09-11T08:26:07.579 INFO:tasks.cephadm:Adding mon.b on smithi172 2024-09-11T08:26:07.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi172:172.21.15.172=b' 2024-09-11T08:26:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:08.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2668224476' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:26:08.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:07 smithi043 ceph-mon[29355]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:08.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:07 smithi043 ceph-mon[29355]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:09.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.483+0000 7f0ad20c2640 1 -- 172.21.15.172:0/618680669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 msgr2=0x7f0acc1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:09.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.483+0000 7f0ad20c2640 1 --2- 172.21.15.172:0/618680669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 0x7f0acc1025b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0abc004850 tx=0x7f0abc009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:09.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.483+0000 7f0ad20c2640 1 -- 172.21.15.172:0/618680669 shutdown_connections 2024-09-11T08:26:09.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.483+0000 7f0ad20c2640 1 --2- 172.21.15.172:0/618680669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 0x7f0acc1025b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:09.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.483+0000 7f0ad20c2640 1 -- 172.21.15.172:0/618680669 >> 172.21.15.172:0/618680669 conn(0x7f0acc0f7d90 msgr2=0x7f0acc0fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:09.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.483+0000 7f0ad20c2640 1 -- 172.21.15.172:0/618680669 shutdown_connections 2024-09-11T08:26:09.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.484+0000 7f0ad20c2640 1 -- 172.21.15.172:0/618680669 wait complete. 2024-09-11T08:26:09.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.484+0000 7f0ad20c2640 1 Processor -- start 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.484+0000 7f0ad20c2640 1 -- start start 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.484+0000 7f0ad20c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 0x7f0acc111c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.484+0000 7f0ad20c2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc1121c0 con 0x7f0acc1021e0 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.484+0000 7f0acb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 0x7f0acc111c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.484+0000 7f0acb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 0x7f0acc111c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.172:47594/0 (socket says 172.21.15.172:47594) 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.484+0000 7f0acb7fe640 1 -- 172.21.15.172:0/3130080040 learned_addr learned my addr 172.21.15.172:0/3130080040 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.485+0000 7f0acb7fe640 1 -- 172.21.15.172:0/3130080040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc005040 con 0x7f0acc1021e0 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.485+0000 7f0acb7fe640 1 --2- 172.21.15.172:0/3130080040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 0x7f0acc111c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0abc003080 tx=0x7f0abc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.485+0000 7f0ac8ff9640 1 -- 172.21.15.172:0/3130080040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0abc038070 con 0x7f0acc1021e0 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.485+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0acc1123c0 con 0x7f0acc1021e0 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.485+0000 7f0ac8ff9640 1 -- 172.21.15.172:0/3130080040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f0abc0093b0 con 0x7f0acc1021e0 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.485+0000 7f0ac8ff9640 1 -- 172.21.15.172:0/3130080040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0abc03c410 con 0x7f0acc1021e0 2024-09-11T08:26:09.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.485+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0acc112830 con 0x7f0acc1021e0 2024-09-11T08:26:09.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.486+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0acc0fd950 con 0x7f0acc1021e0 2024-09-11T08:26:09.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.486+0000 7f0ac8ff9640 1 -- 172.21.15.172:0/3130080040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f0abc030020 con 0x7f0acc1021e0 2024-09-11T08:26:09.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.487+0000 7f0ac8ff9640 1 --2- 172.21.15.172:0/3130080040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f0ab8039510 0x7f0ab803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:09.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.487+0000 7f0ac8ff9640 1 -- 172.21.15.172:0/3130080040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0abc06bc80 con 0x7f0acc1021e0 2024-09-11T08:26:09.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.487+0000 7f0ac3fff640 1 --2- 172.21.15.172:0/3130080040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f0ab8039510 0x7f0ab803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:09.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.487+0000 7f0ac3fff640 1 --2- 172.21.15.172:0/3130080040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f0ab8039510 0x7f0ab803b9c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006fd0 tx=0x7f0ab4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:09.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.492+0000 7f0ac8ff9640 1 -- 172.21.15.172:0/3130080040 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f0abc03bdc0 con 0x7f0acc1021e0 2024-09-11T08:26:09.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.656+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi172:172.21.15.172=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f0acc1059d0 con 0x7f0ab8039510 2024-09-11T08:26:09.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.661+0000 7f0ac8ff9640 1 -- 172.21.15.172:0/3130080040 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0acc1059d0 con 0x7f0ab8039510 2024-09-11T08:26:09.662 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mon update... 2024-09-11T08:26:09.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f0ab8039510 msgr2=0x7f0ab803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 --2- 172.21.15.172:0/3130080040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f0ab8039510 0x7f0ab803b9c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006fd0 tx=0x7f0ab4006db0 comp rx=0 tx=0).stop 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 msgr2=0x7f0acc111c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 --2- 172.21.15.172:0/3130080040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 0x7f0acc111c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0abc003080 tx=0x7f0abc009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 shutdown_connections 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 --2- 172.21.15.172:0/3130080040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f0ab8039510 0x7f0ab803b9c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 --2- 172.21.15.172:0/3130080040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0acc1021e0 0x7f0acc111c80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 >> 172.21.15.172:0/3130080040 conn(0x7f0acc0f7d90 msgr2=0x7f0acc0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 shutdown_connections 2024-09-11T08:26:09.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:09.663+0000 7f0ad20c2640 1 -- 172.21.15.172:0/3130080040 wait complete. 2024-09-11T08:26:10.233 DEBUG:teuthology.orchestra.run.smithi043:mon.c> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T08:26:10.236 DEBUG:teuthology.orchestra.run.smithi172:mon.b> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T08:26:10.238 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:26:10.239 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:26:11.029 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:11.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi172:172.21.15.172=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi172:172.21.15.172=b;count:3 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:10 smithi043 ceph-mon[29355]: Deploying daemon mon.b on smithi172 2024-09-11T08:26:13.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:12 smithi043 ceph-mon[29355]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:14.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.110+0000 7f9233ae4640 1 -- 172.21.15.172:0/3077983353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 msgr2=0x7f922c0fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:14.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.110+0000 7f9233ae4640 1 --2- 172.21.15.172:0/3077983353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 0x7f922c0fd3b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9228004850 tx=0x7f9228009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:14.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.110+0000 7f9233ae4640 1 -- 172.21.15.172:0/3077983353 shutdown_connections 2024-09-11T08:26:14.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.110+0000 7f9233ae4640 1 --2- 172.21.15.172:0/3077983353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 0x7f922c0fd3b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:14.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.110+0000 7f9233ae4640 1 -- 172.21.15.172:0/3077983353 >> 172.21.15.172:0/3077983353 conn(0x7f922c071790 msgr2=0x7f922c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.110+0000 7f9233ae4640 1 -- 172.21.15.172:0/3077983353 shutdown_connections 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.110+0000 7f9233ae4640 1 -- 172.21.15.172:0/3077983353 wait complete. 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.111+0000 7f9233ae4640 1 Processor -- start 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.111+0000 7f9233ae4640 1 -- start start 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.111+0000 7f9233ae4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 0x7f922c113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.111+0000 7f9233ae4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c114470 con 0x7f922c0fcfe0 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.111+0000 7f9231880640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 0x7f922c113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.111+0000 7f9231880640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 0x7f922c113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.172:47602/0 (socket says 172.21.15.172:47602) 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.111+0000 7f9231880640 1 -- 172.21.15.172:0/3909351955 learned_addr learned my addr 172.21.15.172:0/3909351955 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.112+0000 7f9231880640 1 -- 172.21.15.172:0/3909351955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9228005040 con 0x7f922c0fcfe0 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.112+0000 7f9231880640 1 --2- 172.21.15.172:0/3909351955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 0x7f922c113f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9228003080 tx=0x7f9228009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.112+0000 7f92227fc640 1 -- 172.21.15.172:0/3909351955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9228038070 con 0x7f922c0fcfe0 2024-09-11T08:26:14.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.112+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c114670 con 0x7f922c0fcfe0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.112+0000 7f92227fc640 1 -- 172.21.15.172:0/3909351955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f92280093b0 con 0x7f922c0fcfe0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.112+0000 7f92227fc640 1 -- 172.21.15.172:0/3909351955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f922803c410 con 0x7f922c0fcfe0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.112+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c114a60 con 0x7f922c0fcfe0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.113+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c0fb860 con 0x7f922c0fcfe0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.113+0000 7f92227fc640 1 -- 172.21.15.172:0/3909351955 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f9228030020 con 0x7f922c0fcfe0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.113+0000 7f92227fc640 1 --2- 172.21.15.172:0/3909351955 >> v2:172.21.15.43:6800/1424089977 conn(0x7f9218039510 0x7f921803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.113+0000 7f92227fc640 1 -- 172.21.15.172:0/3909351955 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f922806bc80 con 0x7f922c0fcfe0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.113+0000 7f923107f640 1 --2- 172.21.15.172:0/3909351955 >> v2:172.21.15.43:6800/1424089977 conn(0x7f9218039510 0x7f921803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.114+0000 7f923107f640 1 --2- 172.21.15.172:0/3909351955 >> v2:172.21.15.43:6800/1424089977 conn(0x7f9218039510 0x7f921803b9c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f921c006fd0 tx=0x7f921c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.118+0000 7f92227fc640 1 -- 172.21.15.172:0/3909351955 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f922803bdc0 con 0x7f922c0fcfe0 2024-09-11T08:26:14.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.350+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f922c10f5a0 con 0x7f922c0fcfe0 2024-09-11T08:26:14.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.351+0000 7f92227fc640 1 -- 172.21.15.172:0/3909351955 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+731 (secure 0 0 0) 0x7f9228041020 con 0x7f922c0fcfe0 2024-09-11T08:26:14.352 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:26:14.352 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":1,"fsid":"327ee848-7017-11ef-bcea-c7b262605968","modified":"2024-09-11T08:24:40.219887Z","created":"2024-09-11T08:24:40.219887Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T08:26:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 1 2024-09-11T08:26:14.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 >> v2:172.21.15.43:6800/1424089977 conn(0x7f9218039510 msgr2=0x7f921803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:14.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 --2- 172.21.15.172:0/3909351955 >> v2:172.21.15.43:6800/1424089977 conn(0x7f9218039510 0x7f921803b9c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f921c006fd0 tx=0x7f921c006db0 comp rx=0 tx=0).stop 2024-09-11T08:26:14.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 msgr2=0x7f922c113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:14.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 --2- 172.21.15.172:0/3909351955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 0x7f922c113f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9228003080 tx=0x7f9228009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:14.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 shutdown_connections 2024-09-11T08:26:14.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 --2- 172.21.15.172:0/3909351955 >> v2:172.21.15.43:6800/1424089977 conn(0x7f9218039510 0x7f921803b9c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:14.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 --2- 172.21.15.172:0/3909351955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c0fcfe0 0x7f922c113f30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:14.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 >> 172.21.15.172:0/3909351955 conn(0x7f922c071790 msgr2=0x7f922c0f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:14.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 shutdown_connections 2024-09-11T08:26:14.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:14.353+0000 7f9233ae4640 1 -- 172.21.15.172:0/3909351955 wait complete. 2024-09-11T08:26:15.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:14 smithi043 ceph-mon[29355]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:15.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/3909351955' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:26:16.284 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:26:16.285 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:26:16.544 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:26:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: mon.a calling monitor election 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: mon.b calling monitor election 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: monmap e2: 2 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: fsmap 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: mgrmap e13: y(active, since 40s) 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: overall HEALTH_OK 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:20.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:20.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:19 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:21.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.024+0000 7fbe6c365640 1 -- 172.21.15.172:0/1623367428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe500034f0 msgr2=0x7fbe50005970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:21.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.024+0000 7fbe6c365640 1 --2- 172.21.15.172:0/1623367428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe500034f0 0x7fbe50005970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe640fc250 tx=0x7fbe5400b040 comp rx=0 tx=0).stop 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.024+0000 7fbe6c365640 1 -- 172.21.15.172:0/1623367428 shutdown_connections 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.024+0000 7fbe6c365640 1 --2- 172.21.15.172:0/1623367428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe500034f0 0x7fbe50005970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.024+0000 7fbe6c365640 1 -- 172.21.15.172:0/1623367428 >> 172.21.15.172:0/1623367428 conn(0x7fbe640f7ec0 msgr2=0x7fbe640fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.024+0000 7fbe6c365640 1 -- 172.21.15.172:0/1623367428 shutdown_connections 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.024+0000 7fbe6c365640 1 -- 172.21.15.172:0/1623367428 wait complete. 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.025+0000 7fbe6c365640 1 Processor -- start 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.025+0000 7fbe6c365640 1 -- start start 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.025+0000 7fbe6c365640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 0x7fbe64111f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.025+0000 7fbe6c365640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe64112480 0x7fbe6410bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.025+0000 7fbe6c365640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64112a00 con 0x7fbe64112480 2024-09-11T08:26:21.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.025+0000 7fbe6c365640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64112b40 con 0x7fbe500034f0 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe6a101640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 0x7fbe64111f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe6a101640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 0x7fbe64111f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45422/0 (socket says 172.21.15.172:45422) 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe6a101640 1 -- 172.21.15.172:0/2497961283 learned_addr learned my addr 172.21.15.172:0/2497961283 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe69900640 1 --2- 172.21.15.172:0/2497961283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe64112480 0x7fbe6410bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe6a101640 1 -- 172.21.15.172:0/2497961283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 msgr2=0x7fbe64111f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe6a101640 1 -- 172.21.15.172:0/2497961283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 msgr2=0x7fbe64111f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe6a101640 1 --2- 172.21.15.172:0/2497961283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 0x7fbe64111f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe6a101640 1 --2- 172.21.15.172:0/2497961283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 0x7fbe64111f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe69900640 1 -- 172.21.15.172:0/2497961283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 msgr2=0x7fbe64111f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe69900640 1 --2- 172.21.15.172:0/2497961283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 0x7fbe64111f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe69900640 1 -- 172.21.15.172:0/2497961283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe54005040 con 0x7fbe64112480 2024-09-11T08:26:21.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.026+0000 7fbe69900640 1 --2- 172.21.15.172:0/2497961283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe64112480 0x7fbe6410bec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000b580 tx=0x7fbe600090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:21.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.027+0000 7fbe5affd640 1 -- 172.21.15.172:0/2497961283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fbe60011840 con 0x7fbe64112480 2024-09-11T08:26:21.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.027+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6410c460 con 0x7fbe64112480 2024-09-11T08:26:21.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.027+0000 7fbe5affd640 1 -- 172.21.15.172:0/2497961283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fbe60011e10 con 0x7fbe64112480 2024-09-11T08:26:21.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.027+0000 7fbe5affd640 1 -- 172.21.15.172:0/2497961283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fbe60010e00 con 0x7fbe64112480 2024-09-11T08:26:21.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.027+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6410ca00 con 0x7fbe64112480 2024-09-11T08:26:21.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.028+0000 7fbe5affd640 1 -- 172.21.15.172:0/2497961283 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7fbe6001a450 con 0x7fbe64112480 2024-09-11T08:26:21.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.028+0000 7fbe5affd640 1 --2- 172.21.15.172:0/2497961283 >> v2:172.21.15.43:6800/1424089977 conn(0x7fbe48039640 0x7fbe4803baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:21.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.028+0000 7fbe5affd640 1 -- 172.21.15.172:0/2497961283 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fbe6004d060 con 0x7fbe64112480 2024-09-11T08:26:21.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.028+0000 7fbe6a101640 1 --2- 172.21.15.172:0/2497961283 >> v2:172.21.15.43:6800/1424089977 conn(0x7fbe48039640 0x7fbe4803baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:21.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.028+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6410cce0 con 0x7fbe64112480 2024-09-11T08:26:21.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.029+0000 7fbe6a101640 1 --2- 172.21.15.172:0/2497961283 >> v2:172.21.15.43:6800/1424089977 conn(0x7fbe48039640 0x7fbe4803baf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe54004560 tx=0x7fbe5400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:21.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.033+0000 7fbe5affd640 1 -- 172.21.15.172:0/2497961283 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fbe60015030 con 0x7fbe64112480 2024-09-11T08:26:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:20 smithi043 ceph-mon[29355]: Deploying daemon mon.c on smithi043 2024-09-11T08:26:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:20 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:21.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:20 smithi172 ceph-mon[32987]: Deploying daemon mon.c on smithi043 2024-09-11T08:26:21.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:20 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:21.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.250+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbe640fda60 con 0x7fbe64112480 2024-09-11T08:26:21.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.251+0000 7fbe5affd640 1 -- 172.21.15.172:0/2497961283 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1001 (secure 0 0 0) 0x7fbe60017aa0 con 0x7fbe64112480 2024-09-11T08:26:21.252 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:26:21.252 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"fsid":"327ee848-7017-11ef-bcea-c7b262605968","modified":"2024-09-11T08:26:14.773668Z","created":"2024-09-11T08:24:40.219887Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T08:26:21.252 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 2 2024-09-11T08:26:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 >> v2:172.21.15.43:6800/1424089977 conn(0x7fbe48039640 msgr2=0x7fbe4803baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 --2- 172.21.15.172:0/2497961283 >> v2:172.21.15.43:6800/1424089977 conn(0x7fbe48039640 0x7fbe4803baf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe54004560 tx=0x7fbe5400b040 comp rx=0 tx=0).stop 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe64112480 msgr2=0x7fbe6410bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 --2- 172.21.15.172:0/2497961283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe64112480 0x7fbe6410bec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000b580 tx=0x7fbe600090f0 comp rx=0 tx=0).stop 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 shutdown_connections 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 --2- 172.21.15.172:0/2497961283 >> v2:172.21.15.43:6800/1424089977 conn(0x7fbe48039640 0x7fbe4803baf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 --2- 172.21.15.172:0/2497961283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe64112480 0x7fbe6410bec0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 --2- 172.21.15.172:0/2497961283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500034f0 0x7fbe64111f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 >> 172.21.15.172:0/2497961283 conn(0x7fbe640f7ec0 msgr2=0x7fbe640fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 shutdown_connections 2024-09-11T08:26:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:21.253+0000 7fbe6c365640 1 -- 172.21.15.172:0/2497961283 wait complete. 2024-09-11T08:26:22.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/2497961283' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:26:22.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:21 smithi043 ceph-mon[29355]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/2497961283' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:26:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:21 smithi172 ceph-mon[32987]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:22.814 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:26:22.814 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:26:23.067 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:26:23.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:22 smithi043 systemd[1]: Starting Ceph mon.c for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:26:23.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 podman[40554]: 2024-09-11 08:26:22.973126731 +0000 UTC m=+0.016307786 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:26:23.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 podman[40554]: 2024-09-11 08:26:23.15242783 +0000 UTC m=+0.195608884 container create d8c1e4093b5168b56504b2d0f95854b8f147c183357db35f001f5f4e79308f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-c, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T08:26:23.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 podman[40554]: 2024-09-11 08:26:23.346346373 +0000 UTC m=+0.389527429 container init d8c1e4093b5168b56504b2d0f95854b8f147c183357db35f001f5f4e79308f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:26:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 podman[40554]: 2024-09-11 08:26:23.351269179 +0000 UTC m=+0.394450234 container start d8c1e4093b5168b56504b2d0f95854b8f147c183357db35f001f5f4e79308f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731) 2024-09-11T08:26:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:26:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T08:26:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pidfile_write: ignore empty --pid-file 2024-09-11T08:26:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: load: jerasure load: lrc 2024-09-11T08:26:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:26:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:26:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: DB SUMMARY 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: DB Session ID: UTPXRG3UTEATHPGIK48D 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.env: 0x557c3456b440 2024-09-11T08:26:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.fs: Posix File System 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.info_log: 0x557c36052ec0 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.statistics: (nil) 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.use_fsync: 0 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:26:23.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:26:23.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:26:23.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:26:23.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:26:23.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.db_log_dir: 2024-09-11T08:26:23.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-11T08:26:23.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:26:23.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:26:23.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:26:23.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:26:23.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:26:23.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:26:23.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.write_buffer_manager: 0x557c361107b0 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:26:23.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.unordered_write: 0 2024-09-11T08:26:23.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:26:23.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:26:23.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:26:23.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:26:23.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.row_cache: None 2024-09-11T08:26:23.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.wal_filter: None 2024-09-11T08:26:23.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:26:23.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:26:23.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:26:23.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:26:23.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:26:23.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:26:23.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:26:23.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_open_files: -1 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:26:23.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Compression algorithms supported: 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: kZSTD supported: 0 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:26:23.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.merge_operator: 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_filter: None 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557c35ff9a38) 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:26:23.657 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_type: 0 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_index_type: 0 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_shortening: 1 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: hash_index_allow_collision: 1 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: checksum: 1 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: no_block_cache: 0 2024-09-11T08:26:23.658 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache: 0x557c3602f350 2024-09-11T08:26:23.659 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:26:23.659 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_options: 2024-09-11T08:26:23.659 INFO:journalctl@ceph.mon.c.smithi043.stdout: capacity : 536870912 2024-09-11T08:26:23.659 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_shard_bits : 4 2024-09-11T08:26:23.659 INFO:journalctl@ceph.mon.c.smithi043.stdout: strict_capacity_limit : 0 2024-09-11T08:26:23.659 INFO:journalctl@ceph.mon.c.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:26:23.659 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_compressed: (nil) 2024-09-11T08:26:23.659 INFO:journalctl@ceph.mon.c.smithi043.stdout: persistent_cache: (nil) 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size: 4096 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size_deviation: 10 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_restart_interval: 16 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_block_restart_interval: 1 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: metadata_block_size: 4096 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: partition_filters: 0 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: use_delta_encoding: 1 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:26:23.660 INFO:journalctl@ceph.mon.c.smithi043.stdout: whole_key_filtering: 1 2024-09-11T08:26:23.661 INFO:journalctl@ceph.mon.c.smithi043.stdout: verify_compression: 0 2024-09-11T08:26:23.661 INFO:journalctl@ceph.mon.c.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:26:23.661 INFO:journalctl@ceph.mon.c.smithi043.stdout: format_version: 4 2024-09-11T08:26:23.661 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_index_compression: 1 2024-09-11T08:26:23.661 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_align: 0 2024-09-11T08:26:23.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:26:23.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:26:23.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compression: NoCompression 2024-09-11T08:26:23.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:26:23.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:26:23.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:26:23.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.num_levels: 7 2024-09-11T08:26:23.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:26:23.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:26:23.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:26:23.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:26:23.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:26:23.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:26:23.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:26:23.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:26:23.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:26:23.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:26:23.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:26:23.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:26:23.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:26:23.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:26:23.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:26:23.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:26:23.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:26:23.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:26:23.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:26:23.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:26:23.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:26:23.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:26:23.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:26:23.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:26:23.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:26:23.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:26:23.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:26:23.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:26:23.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:26:23.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:26:23.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:26:23.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:26:23.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:26:23.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:26:23.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:26:23.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:26:23.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:26:23.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:26:23.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:26:23.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:26:23.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:26:23.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:26:23.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:26:23.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:26:23.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:26:23.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:26:23.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:26:23.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:26:23.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:26:23.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:26:23.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:26:23.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.ttl: 2592000 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T08:26:23.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043183375775, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043183376100, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726043183, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "ed221a76-0cbd-4e10-b487-9fae7c9738f3", "db_session_id": "UTPXRG3UTEATHPGIK48D"}} 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043183376439, "job": 1, "event": "recovery_finished"} 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x557c36062700 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: DB pointer 0x557c36032000 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:26:23.671 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** DB Stats ** 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-11T08:26:23.672 INFO:journalctl@ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:26:23.673 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-11T08:26:23.674 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:26:23.675 INFO:journalctl@ceph.mon.c.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: using public_addrv [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: starting mon.c rank -1 at public addrs [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] at bind addrs [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(???) e0 preinit fsid 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).mds e1 new map 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: e1 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: legacy client fscid: -1 2024-09-11T08:26:23.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: No filesystems configured 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2668224476' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2668224476' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi172:172.21.15.172=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi172:172.21.15.172=b;count:3 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:23.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: Deploying daemon mon.b on smithi172 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/3909351955' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.a calling monitor election 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:23.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.b calling monitor election 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: monmap e2: 2 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: fsmap 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mgrmap e13: y(active, since 40s) 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: overall HEALTH_OK 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: Deploying daemon mon.c on smithi043 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/2497961283' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 ceph-mon[40592]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 bash[40554]: d8c1e4093b5168b56504b2d0f95854b8f147c183357db35f001f5f4e79308f28 2024-09-11T08:26:23.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:23 smithi043 systemd[1]: Started Ceph mon.c for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:26:28.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:28.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:26:28.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: mon.a calling monitor election 2024-09-11T08:26:28.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: mon.b calling monitor election 2024-09-11T08:26:28.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: fsmap 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:28.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: mgrmap e13: y(active, since 49s) 2024-09-11T08:26:28.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T08:26:28.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T08:26:28.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T08:26:28.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: mon.c (rank 2) addr [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] is down (out of quorum) 2024-09-11T08:26:28.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:28.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:28.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:28.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.869+0000 7f32a41d2640 1 -- 172.21.15.172:0/2516365611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 msgr2=0x7f3288005970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.869+0000 7f32a41d2640 1 --2- 172.21.15.172:0/2516365611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 0x7f3288005970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f329c06f9e0 tx=0x7f328c002040 comp rx=0 tx=0).stop 2024-09-11T08:26:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.869+0000 7f32a41d2640 1 -- 172.21.15.172:0/2516365611 shutdown_connections 2024-09-11T08:26:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.869+0000 7f32a41d2640 1 --2- 172.21.15.172:0/2516365611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 0x7f3288005970 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.869+0000 7f32a41d2640 1 -- 172.21.15.172:0/2516365611 >> 172.21.15.172:0/2516365611 conn(0x7f329c0f7ec0 msgr2=0x7f329c0fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.869+0000 7f32a41d2640 1 -- 172.21.15.172:0/2516365611 shutdown_connections 2024-09-11T08:26:28.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.869+0000 7f32a41d2640 1 -- 172.21.15.172:0/2516365611 wait complete. 2024-09-11T08:26:28.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.870+0000 7f32a41d2640 1 Processor -- start 2024-09-11T08:26:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.870+0000 7f32a41d2640 1 -- start start 2024-09-11T08:26:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.870+0000 7f32a41d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 0x7f329c118510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.870+0000 7f32a41d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 0x7f329c1124b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.870+0000 7f32a41d2640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 0x7f329c112e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a41d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c119060 con 0x7f32880034f0 2024-09-11T08:26:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a41d2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c11f330 con 0x7f329c118a50 2024-09-11T08:26:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a41d2640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c11f470 con 0x7f329c1129f0 2024-09-11T08:26:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a176d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 0x7f329c1124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 0x7f329c118510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 0x7f329c118510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.172:57168/0 (socket says 172.21.15.172:57168) 2024-09-11T08:26:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a176d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 0x7f329c1124b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54754/0 (socket says 172.21.15.172:54754) 2024-09-11T08:26:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 -- 172.21.15.172:0/4184367817 learned_addr learned my addr 172.21.15.172:0/4184367817 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a276f640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 0x7f329c112e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a176d640 1 -- 172.21.15.172:0/4184367817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 msgr2=0x7f329c1124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a176d640 1 -- 172.21.15.172:0/4184367817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 msgr2=0x7f329c1124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a176d640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 0x7f329c1124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a276f640 1 -- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 msgr2=0x7f329c112e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a276f640 1 -- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 msgr2=0x7f329c112e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a276f640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 0x7f329c112e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a276f640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 0x7f329c112e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a176d640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 0x7f329c1124b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 -- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 msgr2=0x7f329c112e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 0x7f329c112e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 -- 172.21.15.172:0/4184367817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 msgr2=0x7f329c1124b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 0x7f329c1124b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 -- 172.21.15.172:0/4184367817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c005040 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a1f6e640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 0x7f329c118510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f328c0043d0 tx=0x7f328c002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32927fc640 1 -- 172.21.15.172:0/4184367817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c004470 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32927fc640 1 -- 172.21.15.172:0/4184367817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f328c0098e0 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.871+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c11f6a0 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.872+0000 7f32927fc640 1 -- 172.21.15.172:0/4184367817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c033670 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.872+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c11fbc0 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.873+0000 7f32927fc640 1 -- 172.21.15.172:0/4184367817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f328c033aa0 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.873+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c0711f0 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.873+0000 7f32927fc640 1 --2- 172.21.15.172:0/4184367817 >> v2:172.21.15.43:6800/1424089977 conn(0x7f32800399e0 0x7f328003be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.873+0000 7f32927fc640 1 -- 172.21.15.172:0/4184367817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f328c06d760 con 0x7f32880034f0 2024-09-11T08:26:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.873+0000 7f32a176d640 1 --2- 172.21.15.172:0/4184367817 >> v2:172.21.15.43:6800/1424089977 conn(0x7f32800399e0 0x7f328003be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.874+0000 7f32a176d640 1 --2- 172.21.15.172:0/4184367817 >> v2:172.21.15.43:6800/1424089977 conn(0x7f32800399e0 0x7f328003be90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3298006fd0 tx=0x7f3298008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:28.878+0000 7f32927fc640 1 -- 172.21.15.172:0/4184367817 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f328c033d50 con 0x7f32880034f0 2024-09-11T08:26:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:26:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: mon.a calling monitor election 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: mon.b calling monitor election 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:26:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: fsmap 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: mgrmap e13: y(active, since 49s) 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: mon.c (rank 2) addr [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] is down (out of quorum) 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:28.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:29.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.103+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f329c0711f0 con 0x7f32880034f0 2024-09-11T08:26:29.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.103+0000 7f32927fc640 1 -- 172.21.15.172:0/4184367817 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1265 (secure 0 0 0) 0x7f328c03a340 con 0x7f32880034f0 2024-09-11T08:26:29.104 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T08:26:29.104 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":3,"fsid":"327ee848-7017-11ef-bcea-c7b262605968","modified":"2024-09-11T08:26:23.398410Z","created":"2024-09-11T08:24:40.219887Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T08:26:29.105 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 3 2024-09-11T08:26:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 >> v2:172.21.15.43:6800/1424089977 conn(0x7f32800399e0 msgr2=0x7f328003be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 --2- 172.21.15.172:0/4184367817 >> v2:172.21.15.43:6800/1424089977 conn(0x7f32800399e0 0x7f328003be90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3298006fd0 tx=0x7f3298008040 comp rx=0 tx=0).stop 2024-09-11T08:26:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 msgr2=0x7f329c118510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 0x7f329c118510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f328c0043d0 tx=0x7f328c002040 comp rx=0 tx=0).stop 2024-09-11T08:26:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 shutdown_connections 2024-09-11T08:26:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 --2- 172.21.15.172:0/4184367817 >> v2:172.21.15.43:6800/1424089977 conn(0x7f32800399e0 0x7f328003be90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f329c1129f0 0x7f329c112e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f329c118a50 0x7f329c1124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:29.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 --2- 172.21.15.172:0/4184367817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32880034f0 0x7f329c118510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:29.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 >> 172.21.15.172:0/4184367817 conn(0x7f329c0f7ec0 msgr2=0x7f329c0fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:29.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 shutdown_connections 2024-09-11T08:26:29.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:29.105+0000 7f32a41d2640 1 -- 172.21.15.172:0/4184367817 wait complete. 2024-09-11T08:26:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:29 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/4184367817' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:26:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:29 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:29 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:29 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/4184367817' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:26:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:29 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:29 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:30.239 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T08:26:30.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T08:26:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:26:30.621 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:30.622 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:26:30.622 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: mon.a calling monitor election 2024-09-11T08:26:30.622 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: mon.b calling monitor election 2024-09-11T08:26:30.622 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:26:30.622 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:30.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:30.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:30.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:30.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:30.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:30.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:30.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:30.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:26:30.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:26:30.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: fsmap 2024-09-11T08:26:30.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:30.625 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: mgrmap e13: y(active, since 49s) 2024-09-11T08:26:30.625 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T08:26:30.625 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T08:26:30.625 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T08:26:30.625 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: mon.c (rank 2) addr [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] is down (out of quorum) 2024-09-11T08:26:30.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:30.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:30.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:30.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:30.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:30.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/4184367817' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:26:30.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:30.627 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:31.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: mon.c calling monitor election 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: Reconfiguring daemon mon.a on smithi043 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: mon.c calling monitor election 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: mon.b calling monitor election 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: mon.a calling monitor election 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: fsmap 2024-09-11T08:26:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:31.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: mgrmap e13: y(active, since 51s) 2024-09-11T08:26:31.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T08:26:31.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: Cluster is now healthy 2024-09-11T08:26:31.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:31 smithi172 ceph-mon[32987]: overall HEALTH_OK 2024-09-11T08:26:31.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: mon.c calling monitor election 2024-09-11T08:26:31.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:31.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:31.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:26:31.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: Reconfiguring daemon mon.a on smithi043 2024-09-11T08:26:31.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: mon.c calling monitor election 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: mon.b calling monitor election 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: mon.a calling monitor election 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: fsmap 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: mgrmap e13: y(active, since 51s) 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: Cluster is now healthy 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[29355]: overall HEALTH_OK 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: mon.c calling monitor election 2024-09-11T08:26:31.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: Reconfiguring daemon mon.a on smithi043 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: mon.c calling monitor election 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: mon.b calling monitor election 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: mon.a calling monitor election 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: fsmap 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: mgrmap e13: y(active, since 51s) 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: Cluster is now healthy 2024-09-11T08:26:31.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:31 smithi043 ceph-mon[40592]: overall HEALTH_OK 2024-09-11T08:26:32.436 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[29355]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:32.436 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[40592]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:32.437 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:32.438 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:26:32.438 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:32.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:32 smithi172 ceph-mon[32987]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:32.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:26:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:26:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:33.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:33 smithi172 ceph-mon[32987]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:26:33.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:33 smithi172 ceph-mon[32987]: Reconfiguring daemon mon.c on smithi043 2024-09-11T08:26:33.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:33 smithi043 ceph-mon[29355]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:26:33.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:33 smithi043 ceph-mon[29355]: Reconfiguring daemon mon.c on smithi043 2024-09-11T08:26:33.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:33 smithi043 ceph-mon[40592]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:26:33.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:33 smithi043 ceph-mon[40592]: Reconfiguring daemon mon.c on smithi043 2024-09-11T08:26:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 -- 172.21.15.43:0/829580812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 msgr2=0x7f6960102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 --2- 172.21.15.43:0/829580812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 0x7f6960102be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f695c004950 tx=0x7f695c009da0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 -- 172.21.15.43:0/829580812 shutdown_connections 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 --2- 172.21.15.43:0/829580812 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69600fb440 0x7f6960109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 --2- 172.21.15.43:0/829580812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69601031b0 0x7f69600fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 --2- 172.21.15.43:0/829580812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 0x7f6960102be0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 -- 172.21.15.43:0/829580812 >> 172.21.15.43:0/829580812 conn(0x7f6960074c80 msgr2=0x7f69600f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 -- 172.21.15.43:0/829580812 shutdown_connections 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 -- 172.21.15.43:0/829580812 wait complete. 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 Processor -- start 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.109+0000 7f6967416640 1 -- start start 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f6967416640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69600fb440 0x7f6960111c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f6967416640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 0x7f69601121a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f6967416640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69601031b0 0x7f696010cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f6967416640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696010d560 con 0x7f6960102810 2024-09-11T08:26:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f6967416640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696010d6d0 con 0x7f69600fb440 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f6967416640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696010d890 con 0x7f69601031b0 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 0x7f69601121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 0x7f69601121a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55046/0 (socket says 172.21.15.43:55046) 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 -- 172.21.15.43:0/3159509628 learned_addr learned my addr 172.21.15.43:0/3159509628 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69651b2640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69600fb440 0x7f6960111c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69659b3640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69601031b0 0x7f696010cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 -- 172.21.15.43:0/3159509628 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69601031b0 msgr2=0x7f696010cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69601031b0 0x7f696010cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 -- 172.21.15.43:0/3159509628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69600fb440 msgr2=0x7f6960111c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69600fb440 0x7f6960111c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 -- 172.21.15.43:0/3159509628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580098b0 con 0x7f6960102810 2024-09-11T08:26:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.110+0000 7f69649b1640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 0x7f69601121a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6950002ad0 tx=0x7f6950004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.111+0000 7f6955ffb640 1 -- 172.21.15.43:0/3159509628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950007c30 con 0x7f6960102810 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.111+0000 7f6955ffb640 1 -- 172.21.15.43:0/3159509628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f695000f040 con 0x7f6960102810 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.111+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695c005040 con 0x7f6960102810 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.111+0000 7f6955ffb640 1 -- 172.21.15.43:0/3159509628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950009340 con 0x7f6960102810 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.111+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6960119bc0 con 0x7f6960102810 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.112+0000 7f6955ffb640 1 -- 172.21.15.43:0/3159509628 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f6950009730 con 0x7f6960102810 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.112+0000 7f6955ffb640 1 --2- 172.21.15.43:0/3159509628 >> v2:172.21.15.43:6800/1424089977 conn(0x7f694c039800 0x7f694c03bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.112+0000 7f6955ffb640 1 -- 172.21.15.43:0/3159509628 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f695004bc20 con 0x7f6960102810 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.112+0000 7f69651b2640 1 --2- 172.21.15.43:0/3159509628 >> v2:172.21.15.43:6800/1424089977 conn(0x7f694c039800 0x7f694c03bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.112+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69600fe030 con 0x7f6960102810 2024-09-11T08:26:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.113+0000 7f69651b2640 1 --2- 172.21.15.43:0/3159509628 >> v2:172.21.15.43:6800/1424089977 conn(0x7f694c039800 0x7f694c03bcb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f696010e710 tx=0x7f695c035040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.117+0000 7f6955ffb640 1 -- 172.21.15.43:0/3159509628 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f695001c400 con 0x7f6960102810 2024-09-11T08:26:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.282+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f696010e3c0 con 0x7f6960102810 2024-09-11T08:26:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.282+0000 7f6955ffb640 1 -- 172.21.15.43:0/3159509628 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f6950011760 con 0x7f6960102810 2024-09-11T08:26:34.284 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:26:34.284 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-11T08:26:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:26:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] 2024-09-11T08:26:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 >> v2:172.21.15.43:6800/1424089977 conn(0x7f694c039800 msgr2=0x7f694c03bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 --2- 172.21.15.43:0/3159509628 >> v2:172.21.15.43:6800/1424089977 conn(0x7f694c039800 0x7f694c03bcb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f696010e710 tx=0x7f695c035040 comp rx=0 tx=0).stop 2024-09-11T08:26:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 msgr2=0x7f69601121a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 0x7f69601121a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6950002ad0 tx=0x7f6950004600 comp rx=0 tx=0).stop 2024-09-11T08:26:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 shutdown_connections 2024-09-11T08:26:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 --2- 172.21.15.43:0/3159509628 >> v2:172.21.15.43:6800/1424089977 conn(0x7f694c039800 0x7f694c03bcb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69601031b0 0x7f696010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6960102810 0x7f69601121a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 --2- 172.21.15.43:0/3159509628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69600fb440 0x7f6960111c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 >> 172.21.15.43:0/3159509628 conn(0x7f6960074c80 msgr2=0x7f6960100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 shutdown_connections 2024-09-11T08:26:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:34.284+0000 7f6967416640 1 -- 172.21.15.43:0/3159509628 wait complete. 2024-09-11T08:26:34.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:34.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: Reconfiguring daemon mon.b on smithi172 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3159509628' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:34.879 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:34.880 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:34.880 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:26:34.880 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:34.880 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:26:34.880 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:34.880 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: Reconfiguring daemon mon.b on smithi172 2024-09-11T08:26:34.880 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3159509628' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:34.904 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T08:26:34.904 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:26:34.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: Reconfiguring daemon mon.b on smithi172 2024-09-11T08:26:34.920 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3159509628' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:34.933 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:26:34.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:26:35.000 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:26:35.000 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:26:35.028 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:26:35.028 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:26:35.093 INFO:tasks.cephadm:Adding mgr.y on smithi043 2024-09-11T08:26:35.093 INFO:tasks.cephadm:Adding mgr.x on smithi172 2024-09-11T08:26:35.093 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi043=y;smithi172=x' 2024-09-11T08:26:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:26:36.158 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:35 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.159 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:35 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.159 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:35 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:36.159 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:35 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:36.159 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:35 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:36.159 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:35 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.159 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:35 smithi172 ceph-mon[32987]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:36.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:36.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[29355]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:36.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:35 smithi043 ceph-mon[40592]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:36.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1035530550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 msgr2=0x7fc5b4103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1035530550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 0x7fc5b4103b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009cd0 tx=0x7fc5ac004d10 comp rx=0 tx=0).stop 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1035530550 shutdown_connections 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1035530550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 0x7fc5b4103b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1035530550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5b40fc900 0x7fc5b40fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1035530550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc5b40fb6d0 0x7fc5b40fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1035530550 >> 172.21.15.172:0/1035530550 conn(0x7fc5b40f6e90 msgr2=0x7fc5b40f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1035530550 shutdown_connections 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.858+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1035530550 wait complete. 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b9f7e640 1 Processor -- start 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b9f7e640 1 -- start start 2024-09-11T08:26:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b9f7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5b40fb6d0 0x7fc5b4066870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b9f7e640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc5b40fc900 0x7fc5b4111920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b9f7e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 0x7fc5b4113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b9f7e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4114600 con 0x7fc5b40fb6d0 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b9f7e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4114770 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b9f7e640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4114930 con 0x7fc5b40fc900 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 0x7fc5b4113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 0x7fc5b4113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33764/0 (socket says 172.21.15.172:33764) 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 -- 172.21.15.172:0/1162798268 learned_addr learned my addr 172.21.15.172:0/1162798268 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b37fe640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5b40fb6d0 0x7fc5b4066870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b2ffd640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc5b40fc900 0x7fc5b4111920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 -- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc5b40fc900 msgr2=0x7fc5b4111920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc5b40fc900 0x7fc5b4111920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 -- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5b40fb6d0 msgr2=0x7fc5b4066870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5b40fb6d0 0x7fc5b4066870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 -- 172.21.15.172:0/1162798268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0098b0 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b2ffd640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc5b40fc900 0x7fc5b4111920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.859+0000 7fc5b3fff640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 0x7fc5b4113e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009040 tx=0x7fc5ac004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.860+0000 7fc59bfff640 1 -- 172.21.15.172:0/1162798268 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac038070 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.860+0000 7fc59bfff640 1 -- 172.21.15.172:0/1162798268 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fc5ac0096e0 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.860+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b4114bb0 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.860+0000 7fc59bfff640 1 -- 172.21.15.172:0/1162798268 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac03cd30 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.860+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b41150d0 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.861+0000 7fc59bfff640 1 -- 172.21.15.172:0/1162798268 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7fc5ac030070 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.861+0000 7fc59bfff640 1 --2- 172.21.15.172:0/1162798268 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc59c039800 0x7fc59c03bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:36.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.861+0000 7fc59bfff640 1 -- 172.21.15.172:0/1162798268 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc5ac06c930 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.861+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b404bf60 con 0x7fc5b40fd2b0 2024-09-11T08:26:36.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.861+0000 7fc5b37fe640 1 --2- 172.21.15.172:0/1162798268 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc59c039800 0x7fc59c03bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:36.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.862+0000 7fc5b37fe640 1 --2- 172.21.15.172:0/1162798268 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc59c039800 0x7fc59c03bcb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40fc760 tx=0x7fc5a400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:36.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:36.866+0000 7fc59bfff640 1 -- 172.21.15.172:0/1162798268 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc5ac03ce90 con 0x7fc5b40fd2b0 2024-09-11T08:26:37.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.029+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi172=x", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5b405db50 con 0x7fc59c039800 2024-09-11T08:26:37.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.035+0000 7fc59bfff640 1 -- 172.21.15.172:0/1162798268 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc5b405db50 con 0x7fc59c039800 2024-09-11T08:26:37.037 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mgr update... 2024-09-11T08:26:37.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc59c039800 msgr2=0x7fc59c03bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:37.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1162798268 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc59c039800 0x7fc59c03bcb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40fc760 tx=0x7fc5a400b040 comp rx=0 tx=0).stop 2024-09-11T08:26:37.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 msgr2=0x7fc5b4113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:37.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 0x7fc5b4113e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009040 tx=0x7fc5ac004d10 comp rx=0 tx=0).stop 2024-09-11T08:26:37.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 shutdown_connections 2024-09-11T08:26:37.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1162798268 >> v2:172.21.15.43:6800/1424089977 conn(0x7fc59c039800 0x7fc59c03bcb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:37.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc5b40fc900 0x7fc5b4111920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:37.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b40fd2b0 0x7fc5b4113e70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:37.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 --2- 172.21.15.172:0/1162798268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5b40fb6d0 0x7fc5b4066870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:37.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 >> 172.21.15.172:0/1162798268 conn(0x7fc5b40f6e90 msgr2=0x7fc5b40f7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:37.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 shutdown_connections 2024-09-11T08:26:37.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:26:37.038+0000 7fc5b9f7e640 1 -- 172.21.15.172:0/1162798268 wait complete. 2024-09-11T08:26:37.630 DEBUG:teuthology.orchestra.run.smithi172:mgr.x> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T08:26:37.632 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T08:26:37.632 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:26:37.633 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:26:37.653 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:26:37.653 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 08:26:22.795034186 +0000 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 08:21:46.784196481 +0000 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 08:21:46.784196481 +0000 2024-09-11T08:26:37.712 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 08:21:46.784196481 +0000 2024-09-11T08:26:37.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:26:37.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-11T08:26:37.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-11T08:26:37.781 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000131282 s, 3.9 MB/s 2024-09-11T08:26:37.782 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:26:37.842 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 08:26:22.796034174 +0000 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 08:21:46.784196481 +0000 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 08:21:46.784196481 +0000 2024-09-11T08:26:37.901 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 08:21:46.784196481 +0000 2024-09-11T08:26:37.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:26:37.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-11T08:26:37.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-11T08:26:37.968 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000125425 s, 4.1 MB/s 2024-09-11T08:26:37.969 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:26:38.029 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-11T08:26:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:26:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:26:38.087 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 961 Links: 1 2024-09-11T08:26:38.087 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:26:38.087 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:26:38.087 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 08:26:22.796034174 +0000 2024-09-11T08:26:38.087 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 08:21:46.783196492 +0000 2024-09-11T08:26:38.087 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 08:21:46.783196492 +0000 2024-09-11T08:26:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 08:21:46.783196492 +0000 2024-09-11T08:26:38.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:26:38.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-11T08:26:38.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-11T08:26:38.153 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000121031 s, 4.2 MB/s 2024-09-11T08:26:38.155 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:26:38.212 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-11T08:26:38.269 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:26:38.269 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:26:38.270 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 964 Links: 1 2024-09-11T08:26:38.270 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:26:38.270 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:26:38.270 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 08:26:22.797034162 +0000 2024-09-11T08:26:38.270 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 08:21:46.783196492 +0000 2024-09-11T08:26:38.270 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 08:21:46.783196492 +0000 2024-09-11T08:26:38.270 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 08:21:46.783196492 +0000 2024-09-11T08:26:38.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:26:38.333 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi172=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:26:38.333 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: Saving service mgr spec with placement smithi043=y;smithi172=x;count:2 2024-09-11T08:26:38.333 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:38.333 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: Deploying daemon mgr.x on smithi172 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[29355]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi172=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:26:38.334 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: Saving service mgr spec with placement smithi043=y;smithi172=x;count:2 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: Deploying daemon mgr.x on smithi172 2024-09-11T08:26:38.335 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:38 smithi043 ceph-mon[40592]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:38.336 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-11T08:26:38.336 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-11T08:26:38.336 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000115343 s, 4.4 MB/s 2024-09-11T08:26:38.337 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:26:38.394 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:26:38.394 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:26:38.410 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:26:38.410 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-09-11T08:26:38.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi172=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:26:38.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: Saving service mgr spec with placement smithi043=y;smithi172=x;count:2 2024-09-11T08:26:38.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:38.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:38.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:38.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: Deploying daemon mgr.x on smithi172 2024-09-11T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:38 smithi172 ceph-mon[32987]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-11 08:26:13.820454712 +0000 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-11 08:21:46.321010542 +0000 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-11 08:21:46.321010542 +0000 2024-09-11T08:26:38.428 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-11 08:21:46.321010542 +0000 2024-09-11T08:26:38.428 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:26:38.493 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-11T08:26:38.493 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-11T08:26:38.493 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00021436 s, 2.4 MB/s 2024-09-11T08:26:38.494 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:26:38.550 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-09-11T08:26:38.605 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:26:38.605 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:26:38.605 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-11T08:26:38.606 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:26:38.606 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:26:38.606 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-11 08:26:13.820454712 +0000 2024-09-11T08:26:38.606 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-11 08:21:46.320010795 +0000 2024-09-11T08:26:38.606 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-11 08:21:46.320010795 +0000 2024-09-11T08:26:38.606 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-11 08:21:46.320010795 +0000 2024-09-11T08:26:38.606 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:26:38.669 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-11T08:26:38.669 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-11T08:26:38.670 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000194715 s, 2.6 MB/s 2024-09-11T08:26:38.671 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:26:38.726 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-09-11T08:26:38.782 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:26:38.782 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:26:38.782 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-11T08:26:38.782 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:26:38.783 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:26:38.783 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-11 08:26:13.821454688 +0000 2024-09-11T08:26:38.783 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-11 08:21:46.318011299 +0000 2024-09-11T08:26:38.783 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-11 08:21:46.318011299 +0000 2024-09-11T08:26:38.783 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-11 08:21:46.318011299 +0000 2024-09-11T08:26:38.783 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:26:38.866 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-11T08:26:38.866 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-11T08:26:38.867 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000261636 s, 2.0 MB/s 2024-09-11T08:26:38.867 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:26:38.924 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-09-11T08:26:38.980 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:26:38.981 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:26:38.981 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-09-11T08:26:38.981 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:26:38.981 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:26:38.981 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-11 08:26:38.819854753 +0000 2024-09-11T08:26:38.981 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-11 08:21:46.320010795 +0000 2024-09-11T08:26:38.981 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-11 08:21:46.320010795 +0000 2024-09-11T08:26:38.981 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-11 08:21:46.320010795 +0000 2024-09-11T08:26:38.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:26:39.061 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-11T08:26:39.061 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-11T08:26:39.061 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00021773 s, 2.4 MB/s 2024-09-11T08:26:39.062 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:26:39.120 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-09-11T08:26:39.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T08:26:39.383 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:26:39.396 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 podman[35062]: 2024-09-11 08:26:39.190239259 +0000 UTC m=+0.015597405 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:26:39.735 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 podman[35062]: 2024-09-11 08:26:39.483895893 +0000 UTC m=+0.309254029 container init f8306e018d2c08d426fb08e674676534ef36c9062d163b8adbfb50b847ee388a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T08:26:39.735 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 podman[35062]: 2024-09-11 08:26:39.48871432 +0000 UTC m=+0.314072463 container start f8306e018d2c08d426fb08e674676534ef36c9062d163b8adbfb50b847ee388a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T08:26:39.736 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 ceph-mgr[35100]: -- 172.21.15.172:0/614878864 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564f15236340 con 0x564f15243000 2024-09-11T08:26:39.736 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 bash[35062]: f8306e018d2c08d426fb08e674676534ef36c9062d163b8adbfb50b847ee388a 2024-09-11T08:26:39.736 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 systemd[1]: Started Ceph mgr.x for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:26:39.736 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:39.606+0000 7f962d78fec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:26:39.736 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:39.665+0000 7f962d78fec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:26:39.736 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:39.733+0000 7f962d78fec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:26:40.014 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:39 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:39.864+0000 7f962d78fec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:26:40.015 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:40.013+0000 7f962d78fec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:26:40.641 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:40.292+0000 7f962d78fec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:26:40.641 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:40.357+0000 7f962d78fec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:26:40.641 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:40.525+0000 7f962d78fec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:26:40.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[40592]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:40.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:40.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[29355]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:40.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:40 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-mon[32987]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:40.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:40.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:40 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:41.162 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T08:26:41.244 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T08:26:41.254 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-09-11T08:26:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-09-11T08:26:41.254 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00869865 s, 1.2 GB/s 2024-09-11T08:26:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-09-11T08:26:41.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:41.383+0000 7f962d78fec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:26:41.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:41.447+0000 7f962d78fec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:26:41.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:41.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:41.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:41.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:41.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:41.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:26:41.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:26:41.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:41.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-09-11T08:26:42.160 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:42.160 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:42.160 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:42.160 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:42.160 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:42.160 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:26:42.160 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:26:42.160 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:42.336 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:26:42.450 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.158+0000 7f962d78fec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:26:42.450 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.270+0000 7f962d78fec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:26:42.450 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.389+0000 7f962d78fec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:26:42.450 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.448+0000 7f962d78fec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:26:42.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.573+0000 7f962d78fec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:26:42.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.639+0000 7f962d78fec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:26:42.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.697+0000 7f962d78fec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:26:43.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:42 smithi043 ceph-mon[40592]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:26:43.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:42 smithi043 ceph-mon[40592]: Reconfiguring daemon mgr.y on smithi043 2024-09-11T08:26:43.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:42 smithi043 ceph-mon[40592]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:43.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:42 smithi043 ceph-mon[29355]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:26:43.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:42 smithi043 ceph-mon[29355]: Reconfiguring daemon mgr.y on smithi043 2024-09-11T08:26:43.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:42 smithi043 ceph-mon[29355]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:43.045 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-mon[32987]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:26:43.045 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-mon[32987]: Reconfiguring daemon mgr.y on smithi043 2024-09-11T08:26:43.045 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-mon[32987]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:43.045 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.833+0000 7f962d78fec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:26:43.045 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.897+0000 7f962d78fec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:26:43.045 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:42.963+0000 7f962d78fec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:26:43.046 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:43.044+0000 7f962d78fec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:26:43.481 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:43.101+0000 7f962d78fec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:26:43.481 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:43.157+0000 7f962d78fec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:26:43.846 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:43.846 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:43.846 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[29355]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:43.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:43 smithi043 ceph-mon[40592]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:44.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:26:43.909+0000 7f962d78fec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:26:44.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:44.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:44.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:44.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:44.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:26:44.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:44.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:43 smithi172 ceph-mon[32987]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 -- 172.21.15.43:0/2332473654 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b440fe4e0 msgr2=0x7f3b44105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/2332473654 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b440fe4e0 0x7f3b44105000 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009d00 tx=0x7f3b3c004d10 comp rx=0 tx=0).stop 2024-09-11T08:26:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 -- 172.21.15.43:0/2332473654 shutdown_connections 2024-09-11T08:26:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/2332473654 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b440fe4e0 0x7f3b44105000 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/2332473654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b440fb530 0x7f3b440fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/2332473654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440fabe0 0x7f3b440faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 -- 172.21.15.43:0/2332473654 >> 172.21.15.43:0/2332473654 conn(0x7f3b440f6640 msgr2=0x7f3b440f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:26:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 -- 172.21.15.43:0/2332473654 shutdown_connections 2024-09-11T08:26:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 -- 172.21.15.43:0/2332473654 wait complete. 2024-09-11T08:26:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 Processor -- start 2024-09-11T08:26:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.310+0000 7f3b4aecd640 1 -- start start 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b4aecd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b440fabe0 0x7f3b440fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b4aecd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440fb530 0x7f3b440fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b4aecd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b440fe4e0 0x7f3b440fc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b4aecd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b440fc860 con 0x7f3b440fabe0 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b4aecd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b440fc9d0 con 0x7f3b440fb530 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b4aecd640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44116db0 con 0x7f3b440fe4e0 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b48c69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b440fabe0 0x7f3b440fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b4946a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b440fe4e0 0x7f3b440fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:44.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440fb530 0x7f3b440fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b48c69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b440fabe0 0x7f3b440fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46738/0 (socket says 172.21.15.43:46738) 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b48c69640 1 -- 172.21.15.43:0/3238441040 learned_addr learned my addr 172.21.15.43:0/3238441040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b43fff640 1 -- 172.21.15.43:0/3238441040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b440fe4e0 msgr2=0x7f3b440fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b43fff640 1 --2- 172.21.15.43:0/3238441040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b440fe4e0 0x7f3b440fc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b43fff640 1 -- 172.21.15.43:0/3238441040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b440fabe0 msgr2=0x7f3b440fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b43fff640 1 --2- 172.21.15.43:0/3238441040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b440fabe0 0x7f3b440fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b43fff640 1 -- 172.21.15.43:0/3238441040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b38005040 con 0x7f3b440fb530 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.311+0000 7f3b43fff640 1 --2- 172.21.15.43:0/3238441040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440fb530 0x7f3b440fbbe0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3b380093d0 tx=0x7f3b38009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.312+0000 7f3b417fa640 1 -- 172.21.15.43:0/3238441040 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3800e6c0 con 0x7f3b440fb530 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.312+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c0098b0 con 0x7f3b440fb530 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.312+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b44117330 con 0x7f3b440fb530 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.312+0000 7f3b417fa640 1 -- 172.21.15.43:0/3238441040 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f3b3800f040 con 0x7f3b440fb530 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.312+0000 7f3b417fa640 1 -- 172.21.15.43:0/3238441040 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380139f0 con 0x7f3b440fb530 2024-09-11T08:26:44.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.313+0000 7f3b417fa640 1 -- 172.21.15.43:0/3238441040 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) v1 ==== 45513+0+0 (secure 0 0 0) 0x7f3b3801c410 con 0x7f3b440fb530 2024-09-11T08:26:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.313+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b30005040 con 0x7f3b440fb530 2024-09-11T08:26:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.313+0000 7f3b417fa640 1 --2- 172.21.15.43:0/3238441040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3b2c039800 0x7f3b2c03bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:26:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.313+0000 7f3b417fa640 1 -- 172.21.15.43:0/3238441040 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3b3804bb20 con 0x7f3b440fb530 2024-09-11T08:26:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.313+0000 7f3b48c69640 1 --2- 172.21.15.43:0/3238441040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3b2c039800 0x7f3b2c03bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:26:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.313+0000 7f3b48c69640 1 --2- 172.21.15.43:0/3238441040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3b2c039800 0x7f3b2c03bcb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b34004a50 tx=0x7f3b3400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:26:44.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.317+0000 7f3b417fa640 1 -- 172.21.15.43:0/3238441040 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3b38022a30 con 0x7f3b440fb530 2024-09-11T08:26:44.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.493+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b30000bf0 con 0x7f3b2c039800 2024-09-11T08:26:44.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:26:44.780+0000 7f3b417fa640 1 -- 172.21.15.43:0/3238441040 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 14) v1 ==== 90689+0+0 (secure 0 0 0) 0x7f3b380112c0 con 0x7f3b440fb530 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: Standby manager daemon x started 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:26:45.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: Standby manager daemon x started 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:26:45.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:44 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:45.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: Standby manager daemon x started 2024-09-11T08:26:45.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:26:45.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:26:45.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:26:45.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='mgr.? 172.21.15.172:0/4126406529' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:26:45.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:45.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:26:45.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:26:45.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:26:45.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:44 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:45 smithi043 ceph-mon[29355]: mgrmap e14: y(active, since 65s), standbys: x 2024-09-11T08:26:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:45 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:26:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:45 smithi043 ceph-mon[29355]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:45 smithi043 ceph-mon[40592]: mgrmap e14: y(active, since 65s), standbys: x 2024-09-11T08:26:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:45 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:26:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:45 smithi043 ceph-mon[40592]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:45 smithi172 ceph-mon[32987]: mgrmap e14: y(active, since 65s), standbys: x 2024-09-11T08:26:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:45 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:26:46.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:45 smithi172 ceph-mon[32987]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:47.165 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2964690956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]: dispatch 2024-09-11T08:26:47.165 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]: dispatch 2024-09-11T08:26:47.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]': finished 2024-09-11T08:26:47.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[40592]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:26:47.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:26:47.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2964690956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]: dispatch 2024-09-11T08:26:47.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]: dispatch 2024-09-11T08:26:47.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]': finished 2024-09-11T08:26:47.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[29355]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:26:47.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:46 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:26:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2964690956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]: dispatch 2024-09-11T08:26:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]: dispatch 2024-09-11T08:26:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0752b9ca-c21a-45f5-9974-5942fd4e17a7"}]': finished 2024-09-11T08:26:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:46 smithi172 ceph-mon[32987]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:26:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:46 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:26:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:47 smithi172 ceph-mon[32987]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3499696850' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:26:48.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:47 smithi043 ceph-mon[29355]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:48.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3499696850' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:26:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:47 smithi043 ceph-mon[40592]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3499696850' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:26:50.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:50 smithi043 ceph-mon[29355]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:50 smithi043 ceph-mon[40592]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:50 smithi172 ceph-mon[32987]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:52 smithi043 ceph-mon[29355]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:52.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:52 smithi043 ceph-mon[40592]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:52 smithi172 ceph-mon[32987]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:53.514 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:53 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:26:53.514 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:53 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:53.514 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:53 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:26:53.514 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:53 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:53 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:26:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:53 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:26:54.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:54 smithi043 ceph-mon[40592]: Deploying daemon osd.0 on smithi043 2024-09-11T08:26:54.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:54 smithi043 ceph-mon[40592]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:54 smithi043 ceph-mon[29355]: Deploying daemon osd.0 on smithi043 2024-09-11T08:26:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:54 smithi043 ceph-mon[29355]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:54 smithi172 ceph-mon[32987]: Deploying daemon osd.0 on smithi043 2024-09-11T08:26:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:54 smithi172 ceph-mon[32987]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:56 smithi043 ceph-mon[29355]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:56 smithi043 ceph-mon[40592]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:56.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:56 smithi172 ceph-mon[32987]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:58.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:58 smithi043 ceph-mon[29355]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:58.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:58 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:58.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:58 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:26:58 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:58 smithi043 ceph-mon[40592]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:58 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:58 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:26:58 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:58 smithi172 ceph-mon[32987]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:26:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:58 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:26:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:58 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:26:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:26:58 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.526 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[29355]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:00.526 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[40592]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:00.527 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:00 smithi172 ceph-mon[32987]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.818+0000 7f3b417fa640 1 -- 172.21.15.43:0/3238441040 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3b30000bf0 con 0x7f3b2c039800 2024-09-11T08:27:00.818 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-09-11T08:27:00.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3b2c039800 msgr2=0x7f3b2c03bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:00.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/3238441040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3b2c039800 0x7f3b2c03bcb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b34004a50 tx=0x7f3b3400b040 comp rx=0 tx=0).stop 2024-09-11T08:27:00.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440fb530 msgr2=0x7f3b440fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:00.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/3238441040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440fb530 0x7f3b440fbbe0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3b380093d0 tx=0x7f3b38009da0 comp rx=0 tx=0).stop 2024-09-11T08:27:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 shutdown_connections 2024-09-11T08:27:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/3238441040 >> v2:172.21.15.43:6800/1424089977 conn(0x7f3b2c039800 0x7f3b2c03bcb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/3238441040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b440fe4e0 0x7f3b440fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/3238441040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440fb530 0x7f3b440fbbe0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 --2- 172.21.15.43:0/3238441040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b440fabe0 0x7f3b440fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 >> 172.21.15.43:0/3238441040 conn(0x7f3b440f6640 msgr2=0x7f3b440f8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:27:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 shutdown_connections 2024-09-11T08:27:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:00.820+0000 7f3b4aecd640 1 -- 172.21.15.43:0/3238441040 wait complete. 2024-09-11T08:27:01.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[29355]: from='osd.0 v2:172.21.15.43:6801/3394646540' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[29355]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[40592]: from='osd.0 v2:172.21.15.43:6801/3394646540' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:27:01.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:01 smithi043 ceph-mon[40592]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:27:01.455 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T08:27:01.458 INFO:tasks.cephadm:Deploying osd.1 on smithi043 with /dev/vg_nvme/lv_3... 2024-09-11T08:27:01.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T08:27:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:01 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:01 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:01 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:01.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:01 smithi172 ceph-mon[32987]: from='osd.0 v2:172.21.15.43:6801/3394646540' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:27:01.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:01 smithi172 ceph-mon[32987]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:27:02.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[29355]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:02.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[29355]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:27:02.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[29355]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:27:02.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:02.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[29355]: from='osd.0 v2:172.21.15.43:6801/3394646540' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:02.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[29355]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:02.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[40592]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:02.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[40592]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:27:02.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[40592]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:27:02.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:02.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[40592]: from='osd.0 v2:172.21.15.43:6801/3394646540' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:02.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-mon[40592]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:02 smithi172 ceph-mon[32987]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:02 smithi172 ceph-mon[32987]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:27:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:02 smithi172 ceph-mon[32987]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:27:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:02 smithi172 ceph-mon[32987]: from='osd.0 v2:172.21.15.43:6801/3394646540' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:02 smithi172 ceph-mon[32987]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:02.974 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0[45014]: 2024-09-11T08:27:02.830+0000 7f4c09f15640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T08:27:02.974 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:02 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0[45014]: 2024-09-11T08:27:02.833+0000 7f4c06d50640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:27:03.062 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:27:04.108 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[29355]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:04.108 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[29355]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:27:04.108 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:04.108 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:04.108 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[29355]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:04.108 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[40592]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:04.108 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[40592]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:27:04.109 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:04.109 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:04.109 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:03 smithi043 ceph-mon[40592]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:04.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:03 smithi172 ceph-mon[32987]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:04.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:03 smithi172 ceph-mon[32987]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:27:04.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:03 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:04.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:03 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:04.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:03 smithi172 ceph-mon[32987]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:27:05.109 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: purged_snaps scrub starts 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: purged_snaps scrub ok 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: osd.0 v2:172.21.15.43:6801/3394646540 boot 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: Detected new or changed devices on smithi043 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-11T08:27:05.110 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: purged_snaps scrub starts 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: purged_snaps scrub ok 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: osd.0 v2:172.21.15.43:6801/3394646540 boot 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: Detected new or changed devices on smithi043 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:05.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-11T08:27:05.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:05.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:05.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: purged_snaps scrub starts 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: purged_snaps scrub ok 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: osd.0 v2:172.21.15.43:6801/3394646540 boot 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: Detected new or changed devices on smithi043 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:05.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-11T08:27:05.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:05.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:05.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:05.483 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T08:27:05.558 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T08:27:05.569 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-09-11T08:27:05.569 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-09-11T08:27:05.569 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00956007 s, 1.1 GB/s 2024-09-11T08:27:05.569 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-09-11T08:27:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:05 smithi043 ceph-mon[29355]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:27:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:05 smithi043 ceph-mon[29355]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:06.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:05 smithi043 ceph-mon[40592]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:27:06.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:05 smithi043 ceph-mon[40592]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:06.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-09-11T08:27:06.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:05 smithi172 ceph-mon[32987]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:27:06.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:05 smithi172 ceph-mon[32987]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 -- 172.21.15.43:0/1898187556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fe430 msgr2=0x7f24b0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 --2- 172.21.15.43:0/1898187556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fe430 0x7f24b0104cd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009cd0 tx=0x7f24a8004d10 comp rx=0 tx=0).stop 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 -- 172.21.15.43:0/1898187556 shutdown_connections 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 --2- 172.21.15.43:0/1898187556 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24b00fda80 0x7f24b00fdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 --2- 172.21.15.43:0/1898187556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fe430 0x7f24b0104cd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 --2- 172.21.15.43:0/1898187556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24b00fc7b0 0x7f24b00fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 -- 172.21.15.43:0/1898187556 >> 172.21.15.43:0/1898187556 conn(0x7f24b00f7f90 msgr2=0x7f24b00fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 -- 172.21.15.43:0/1898187556 shutdown_connections 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 -- 172.21.15.43:0/1898187556 wait complete. 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 Processor -- start 2024-09-11T08:27:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.033+0000 7f24b8748640 1 -- start start 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b8748640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fc7b0 0x7f24b010ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b8748640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24b00fda80 0x7f24b010f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b8748640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24b00fe430 0x7f24b0116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b8748640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b0116730 con 0x7f24b00fc7b0 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b8748640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b01168a0 con 0x7f24b00fe430 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b8748640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b0116a60 con 0x7f24b00fda80 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b64e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fc7b0 0x7f24b010ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24b00fda80 0x7f24b010f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24b00fda80 0x7f24b010f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:56982/0 (socket says 172.21.15.43:56982) 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 -- 172.21.15.43:0/3277756130 learned_addr learned my addr 172.21.15.43:0/3277756130 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b64e4640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fc7b0 0x7f24b010ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57894/0 (socket says 172.21.15.43:57894) 2024-09-11T08:27:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b6ce5640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24b00fe430 0x7f24b0116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 -- 172.21.15.43:0/3277756130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24b00fe430 msgr2=0x7f24b0116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24b00fe430 0x7f24b0116160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 -- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fc7b0 msgr2=0x7f24b010ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fc7b0 0x7f24b010ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 -- 172.21.15.43:0/3277756130 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a0005040 con 0x7f24b00fda80 2024-09-11T08:27:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b64e4640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fc7b0 0x7f24b010ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:27:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.034+0000 7f24b5ce3640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24b00fda80 0x7f24b010f100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009780 tx=0x7f24a0009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:27:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.035+0000 7f24a6ffd640 1 -- 172.21.15.43:0/3277756130 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00099c0 con 0x7f24b00fda80 2024-09-11T08:27:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.035+0000 7f24a6ffd640 1 -- 172.21.15.43:0/3277756130 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f24a000f040 con 0x7f24b00fda80 2024-09-11T08:27:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.035+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a80098b0 con 0x7f24b00fda80 2024-09-11T08:27:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.035+0000 7f24a6ffd640 1 -- 172.21.15.43:0/3277756130 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0013ca0 con 0x7f24b00fda80 2024-09-11T08:27:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.035+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b0117020 con 0x7f24b00fda80 2024-09-11T08:27:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.036+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b004bf60 con 0x7f24b00fda80 2024-09-11T08:27:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.036+0000 7f24a6ffd640 1 -- 172.21.15.43:0/3277756130 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 14) v1 ==== 90689+0+0 (secure 0 0 0) 0x7f24a000ebb0 con 0x7f24b00fda80 2024-09-11T08:27:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.037+0000 7f24a6ffd640 1 --2- 172.21.15.43:0/3277756130 >> v2:172.21.15.43:6800/1424089977 conn(0x7f249c06ee30 0x7f249c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.037+0000 7f24a6ffd640 1 -- 172.21.15.43:0/3277756130 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1537+0+0 (secure 0 0 0) 0x7f24a008cf10 con 0x7f24b00fda80 2024-09-11T08:27:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.037+0000 7f24b64e4640 1 --2- 172.21.15.43:0/3277756130 >> v2:172.21.15.43:6800/1424089977 conn(0x7f249c06ee30 0x7f249c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.037+0000 7f24b64e4640 1 --2- 172.21.15.43:0/3277756130 >> v2:172.21.15.43:6800/1424089977 conn(0x7f249c06ee30 0x7f249c0712e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0041d0 tx=0x7f24ac004330 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:27:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.042+0000 7f24a6ffd640 1 -- 172.21.15.43:0/3277756130 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f24a005b110 con 0x7f24b00fda80 2024-09-11T08:27:08.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:08.216+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b0070950 con 0x7f249c06ee30 2024-09-11T08:27:08.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:08 smithi043 ceph-mon[29355]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:08.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:08 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:08.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:08 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:08.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:08 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:08.618 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:08 smithi043 ceph-mon[40592]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:08.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:08 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:08.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:08 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:08.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:08 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:08 smithi172 ceph-mon[32987]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:08 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:08 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:08.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:08 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:09.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:09 smithi043 ceph-mon[29355]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:09.574 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:09 smithi043 ceph-mon[40592]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:09 smithi172 ceph-mon[32987]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:10.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:10 smithi043 ceph-mon[29355]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:10.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:10 smithi043 ceph-mon[40592]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:10 smithi172 ceph-mon[32987]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:11.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3897262787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039df926-3c4b-48cd-9141-3a7684037575"}]: dispatch 2024-09-11T08:27:11.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3897262787' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "039df926-3c4b-48cd-9141-3a7684037575"}]': finished 2024-09-11T08:27:11.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[40592]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:27:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/589580794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:27:11.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3897262787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039df926-3c4b-48cd-9141-3a7684037575"}]: dispatch 2024-09-11T08:27:11.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3897262787' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "039df926-3c4b-48cd-9141-3a7684037575"}]': finished 2024-09-11T08:27:11.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[29355]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:27:11.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:11.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/589580794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:27:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3897262787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "039df926-3c4b-48cd-9141-3a7684037575"}]: dispatch 2024-09-11T08:27:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3897262787' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "039df926-3c4b-48cd-9141-3a7684037575"}]': finished 2024-09-11T08:27:11.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:11 smithi172 ceph-mon[32987]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:27:11.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:11.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/589580794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:27:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:12 smithi043 ceph-mon[40592]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:12 smithi043 ceph-mon[29355]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:12 smithi172 ceph-mon[32987]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:14.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:14 smithi043 ceph-mon[29355]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:14.704 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:14 smithi043 ceph-mon[40592]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:14 smithi172 ceph-mon[32987]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:16.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:16 smithi043 ceph-mon[29355]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:16 smithi043 ceph-mon[40592]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:16 smithi172 ceph-mon[32987]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:17.810 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:17 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:27:17.810 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:17 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:17.811 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:17 smithi043 ceph-mon[40592]: Deploying daemon osd.1 on smithi043 2024-09-11T08:27:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:17 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:27:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:17 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:17 smithi043 ceph-mon[29355]: Deploying daemon osd.1 on smithi043 2024-09-11T08:27:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:17 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:27:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:17 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:17 smithi172 ceph-mon[32987]: Deploying daemon osd.1 on smithi043 2024-09-11T08:27:18.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:18 smithi043 ceph-mon[29355]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:18.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:18 smithi043 ceph-mon[40592]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:18.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:18 smithi172 ceph-mon[32987]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:20.775 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:20 smithi043 ceph-mon[29355]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:20.775 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:20 smithi043 ceph-mon[40592]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:20.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:20 smithi172 ceph-mon[32987]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:22 smithi043 ceph-mon[29355]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:22 smithi043 ceph-mon[40592]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:22 smithi172 ceph-mon[32987]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:22.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[29355]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:24.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:24.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:24.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.756 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[40592]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:24.756 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.756 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.756 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:24.756 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:24.756 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:24 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:24 smithi172 ceph-mon[32987]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:24 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:24 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:24 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:24 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:24 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:25.144 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 1 on host 'smithi043' 2024-09-11T08:27:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.143+0000 7f24a6ffd640 1 -- 172.21.15.43:0/3277756130 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f24b0070950 con 0x7f249c06ee30 2024-09-11T08:27:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 >> v2:172.21.15.43:6800/1424089977 conn(0x7f249c06ee30 msgr2=0x7f249c0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 --2- 172.21.15.43:0/3277756130 >> v2:172.21.15.43:6800/1424089977 conn(0x7f249c06ee30 0x7f249c0712e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0041d0 tx=0x7f24ac004330 comp rx=0 tx=0).stop 2024-09-11T08:27:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24b00fda80 msgr2=0x7f24b010f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24b00fda80 0x7f24b010f100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009780 tx=0x7f24a0009da0 comp rx=0 tx=0).stop 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 shutdown_connections 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 --2- 172.21.15.43:0/3277756130 >> v2:172.21.15.43:6800/1424089977 conn(0x7f249c06ee30 0x7f249c0712e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24b00fda80 0x7f24b010f100 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24b00fe430 0x7f24b0116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 --2- 172.21.15.43:0/3277756130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24b00fc7b0 0x7f24b010ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 >> 172.21.15.43:0/3277756130 conn(0x7f24b00f7f90 msgr2=0x7f24b00f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 shutdown_connections 2024-09-11T08:27:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:25.146+0000 7f24b8748640 1 -- 172.21.15.43:0/3277756130 wait complete. 2024-09-11T08:27:25.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[29355]: from='osd.1 v2:172.21.15.43:6805/2076458881' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:27:25.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[29355]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:27:25.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:25.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:25.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:25.767 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[40592]: from='osd.1 v2:172.21.15.43:6805/2076458881' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:27:25.768 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[40592]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:27:25.768 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:25.768 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:25.768 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:25 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:25.769 DEBUG:teuthology.orchestra.run.smithi043:osd.1> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T08:27:25.772 INFO:tasks.cephadm:Deploying osd.2 on smithi043 with /dev/vg_nvme/lv_2... 2024-09-11T08:27:25.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T08:27:25.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:25 smithi172 ceph-mon[32987]: from='osd.1 v2:172.21.15.43:6805/2076458881' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:27:25.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:25 smithi172 ceph-mon[32987]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:27:25.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:25 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:25.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:25 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:25.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:25 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:27:26.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[29355]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[29355]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[29355]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[29355]: from='osd.1 v2:172.21.15.43:6805/2076458881' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[29355]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[40592]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[40592]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[40592]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:27:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:26.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[40592]: from='osd.1 v2:172.21.15.43:6805/2076458881' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:26.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-mon[40592]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:26.646 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:27:26.593+0000 7f5ad7e25640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T08:27:26.646 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:26 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:27:26.596+0000 7f5ad2c5c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:27:26.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:26 smithi172 ceph-mon[32987]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:26 smithi172 ceph-mon[32987]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:27:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:26 smithi172 ceph-mon[32987]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:27:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:26 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:26 smithi172 ceph-mon[32987]: from='osd.1 v2:172.21.15.43:6805/2076458881' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:26 smithi172 ceph-mon[32987]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[29355]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[29355]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:27:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[40592]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[40592]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:27:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:27 smithi172 ceph-mon[32987]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:27 smithi172 ceph-mon[32987]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:27:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[29355]: purged_snaps scrub starts 2024-09-11T08:27:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[29355]: purged_snaps scrub ok 2024-09-11T08:27:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[29355]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[29355]: osd.1 v2:172.21.15.43:6805/2076458881 boot 2024-09-11T08:27:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[29355]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:27:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[40592]: purged_snaps scrub starts 2024-09-11T08:27:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[40592]: purged_snaps scrub ok 2024-09-11T08:27:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[40592]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[40592]: osd.1 v2:172.21.15.43:6805/2076458881 boot 2024-09-11T08:27:28.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[40592]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:27:28.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:28 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:28 smithi172 ceph-mon[32987]: purged_snaps scrub starts 2024-09-11T08:27:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:28 smithi172 ceph-mon[32987]: purged_snaps scrub ok 2024-09-11T08:27:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:28 smithi172 ceph-mon[32987]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:27:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:28 smithi172 ceph-mon[32987]: osd.1 v2:172.21.15.43:6805/2076458881 boot 2024-09-11T08:27:28.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:28 smithi172 ceph-mon[32987]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:27:28.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:27:29.304 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T08:27:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T08:27:29.409 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-09-11T08:27:29.409 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-09-11T08:27:29.409 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00990765 s, 1.1 GB/s 2024-09-11T08:27:29.410 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-09-11T08:27:29.887 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:29 smithi043 ceph-mon[40592]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:27:29.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:29 smithi043 ceph-mon[29355]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:27:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:29 smithi172 ceph-mon[32987]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:27:30.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-09-11T08:27:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:27:30.840 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:30.840 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: Detected new or changed devices on smithi043 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: Detected new or changed devices on smithi043 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:30.842 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:30 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: Detected new or changed devices on smithi043 2024-09-11T08:27:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-11T08:27:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:30 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 -- 172.21.15.43:0/2150502939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fe770 msgr2=0x7f25040fa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:32.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 --2- 172.21.15.43:0/2150502939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fe770 0x7f25040fa8d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24f8004980 tx=0x7f24f8009da0 comp rx=0 tx=0).stop 2024-09-11T08:27:32.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 -- 172.21.15.43:0/2150502939 shutdown_connections 2024-09-11T08:27:32.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 --2- 172.21.15.43:0/2150502939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25040fb000 0x7f2504104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:32.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 --2- 172.21.15.43:0/2150502939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fe770 0x7f25040fa8d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:32.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 --2- 172.21.15.43:0/2150502939 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f25040fde20 0x7f25040fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:32.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 -- 172.21.15.43:0/2150502939 >> 172.21.15.43:0/2150502939 conn(0x7f25040f6640 msgr2=0x7f25040f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:27:32.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 -- 172.21.15.43:0/2150502939 shutdown_connections 2024-09-11T08:27:32.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 -- 172.21.15.43:0/2150502939 wait complete. 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 Processor -- start 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.060+0000 7f250b06d640 1 -- start start 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f250b06d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fb000 0x7f250410ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f250b06d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25040fde20 0x7f250410f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f250b06d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f25040fe770 0x7f25041164a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f250b06d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504116a70 con 0x7f25040fb000 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f250b06d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504116be0 con 0x7f25040fde20 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f250b06d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2504116da0 con 0x7f25040fe770 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fb000 0x7f250410ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f250960a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f25040fe770 0x7f25041164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:32.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fb000 0x7f250410ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42882/0 (socket says 172.21.15.43:42882) 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 -- 172.21.15.43:0/3801864373 learned_addr learned my addr 172.21.15.43:0/3801864373 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2503fff640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25040fde20 0x7f250410f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 -- 172.21.15.43:0/3801864373 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f25040fe770 msgr2=0x7f25041164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f25040fe770 0x7f25041164a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 -- 172.21.15.43:0/3801864373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25040fde20 msgr2=0x7f250410f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25040fde20 0x7f250410f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 -- 172.21.15.43:0/3801864373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4005040 con 0x7f25040fb000 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f250960a640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f25040fe770 0x7f25041164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:27:32.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2503fff640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25040fde20 0x7f250410f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:27:32.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.061+0000 7f2508e09640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fb000 0x7f250410ef00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009780 tx=0x7f24f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:27:32.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.062+0000 7f25017fa640 1 -- 172.21.15.43:0/3801864373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f40099c0 con 0x7f25040fb000 2024-09-11T08:27:32.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.062+0000 7f25017fa640 1 -- 172.21.15.43:0/3801864373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f24f400f040 con 0x7f25040fb000 2024-09-11T08:27:32.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.062+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f8005040 con 0x7f25040fb000 2024-09-11T08:27:32.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.062+0000 7f25017fa640 1 -- 172.21.15.43:0/3801864373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4013ca0 con 0x7f25040fb000 2024-09-11T08:27:32.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.062+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25041173e0 con 0x7f25040fb000 2024-09-11T08:27:32.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.063+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250404bf60 con 0x7f25040fb000 2024-09-11T08:27:32.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.063+0000 7f25017fa640 1 -- 172.21.15.43:0/3801864373 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 90689+0+0 (secure 0 0 0) 0x7f24f400ebb0 con 0x7f25040fb000 2024-09-11T08:27:32.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.064+0000 7f25017fa640 1 --2- 172.21.15.43:0/3801864373 >> v2:172.21.15.43:6800/1424089977 conn(0x7f24ec06ee30 0x7f24ec0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:32.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.064+0000 7f25017fa640 1 -- 172.21.15.43:0/3801864373 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 1829+0+0 (secure 0 0 0) 0x7f24f408d180 con 0x7f25040fb000 2024-09-11T08:27:32.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.064+0000 7f2503fff640 1 --2- 172.21.15.43:0/3801864373 >> v2:172.21.15.43:6800/1424089977 conn(0x7f24ec06ee30 0x7f24ec0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:32.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.064+0000 7f2503fff640 1 --2- 172.21.15.43:0/3801864373 >> v2:172.21.15.43:6800/1424089977 conn(0x7f24ec06ee30 0x7f24ec0712e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2504110400 tx=0x7f24f8035040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:27:32.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.068+0000 7f25017fa640 1 -- 172.21.15.43:0/3801864373 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f24f405b110 con 0x7f25040fb000 2024-09-11T08:27:32.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:32.242+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f25040fc9f0 con 0x7f24ec06ee30 2024-09-11T08:27:32.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:32 smithi043 ceph-mon[29355]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:32.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:32.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:32.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:32.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:32 smithi043 ceph-mon[40592]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:32.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:32.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:32.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:32.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:32 smithi172 ceph-mon[32987]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:32.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:32.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:32.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:33.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:33 smithi043 ceph-mon[40592]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:33.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:33 smithi043 ceph-mon[29355]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:33 smithi172 ceph-mon[32987]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:34.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[40592]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3254146436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]: dispatch 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]: dispatch 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]': finished 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[40592]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[29355]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3254146436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]: dispatch 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]: dispatch 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]': finished 2024-09-11T08:27:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[29355]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:27:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:34 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:34 smithi172 ceph-mon[32987]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3254146436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]: dispatch 2024-09-11T08:27:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]: dispatch 2024-09-11T08:27:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d56b0955-f9bc-40f9-9fca-78ef5af1fd77"}]': finished 2024-09-11T08:27:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:34 smithi172 ceph-mon[32987]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:27:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:34 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4050530865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:27:35.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4050530865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:27:35.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4050530865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:27:36.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:36 smithi043 ceph-mon[29355]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:36 smithi043 ceph-mon[40592]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:36 smithi172 ceph-mon[32987]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:38.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:38 smithi043 ceph-mon[29355]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:38.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:38 smithi043 ceph-mon[40592]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:38 smithi172 ceph-mon[32987]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:40.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:40 smithi043 ceph-mon[29355]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:40.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:40 smithi043 ceph-mon[40592]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:40 smithi172 ceph-mon[32987]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:27:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:41 smithi043 ceph-mon[29355]: Deploying daemon osd.2 on smithi043 2024-09-11T08:27:41.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:27:41.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:41.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:41 smithi043 ceph-mon[40592]: Deploying daemon osd.2 on smithi043 2024-09-11T08:27:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:27:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:41 smithi172 ceph-mon[32987]: Deploying daemon osd.2 on smithi043 2024-09-11T08:27:42.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:42 smithi043 ceph-mon[29355]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:42.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:42 smithi043 ceph-mon[40592]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:42 smithi172 ceph-mon[32987]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:44 smithi043 ceph-mon[29355]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:44 smithi043 ceph-mon[40592]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:44 smithi172 ceph-mon[32987]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:45.681 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:45 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:45.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:45 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:45.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:45 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:45.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:45 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:45.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:45 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:45.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:45 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:45.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:45 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:45.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:45 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:45.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:45 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:46 smithi043 ceph-mon[29355]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:46.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:46 smithi043 ceph-mon[40592]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:46 smithi172 ceph-mon[32987]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:48.808 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:48 smithi043 ceph-mon[29355]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:48.808 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:48 smithi043 ceph-mon[40592]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.844+0000 7f25017fa640 1 -- 172.21.15.43:0/3801864373 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f25040fc9f0 con 0x7f24ec06ee30 2024-09-11T08:27:48.845 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 2 on host 'smithi043' 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 >> v2:172.21.15.43:6800/1424089977 conn(0x7f24ec06ee30 msgr2=0x7f24ec0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 --2- 172.21.15.43:0/3801864373 >> v2:172.21.15.43:6800/1424089977 conn(0x7f24ec06ee30 0x7f24ec0712e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2504110400 tx=0x7f24f8035040 comp rx=0 tx=0).stop 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fb000 msgr2=0x7f250410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fb000 0x7f250410ef00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009780 tx=0x7f24f4009da0 comp rx=0 tx=0).stop 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 shutdown_connections 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 --2- 172.21.15.43:0/3801864373 >> v2:172.21.15.43:6800/1424089977 conn(0x7f24ec06ee30 0x7f24ec0712e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f25040fe770 0x7f25041164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25040fde20 0x7f250410f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 --2- 172.21.15.43:0/3801864373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25040fb000 0x7f250410ef00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 >> 172.21.15.43:0/3801864373 conn(0x7f25040f6640 msgr2=0x7f25040f8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 shutdown_connections 2024-09-11T08:27:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:48.847+0000 7f250b06d640 1 -- 172.21.15.43:0/3801864373 wait complete. 2024-09-11T08:27:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:48 smithi172 ceph-mon[32987]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:49.471 DEBUG:teuthology.orchestra.run.smithi043:osd.2> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T08:27:49.474 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_1... 2024-09-11T08:27:49.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T08:27:49.801 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:27:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:49 smithi043 ceph-mon[40592]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:27:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:49 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:49 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:49 smithi043 ceph-mon[29355]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:27:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:49 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:49 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:49 smithi172 ceph-mon[32987]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:27:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:49 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:49 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:50.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[29355]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:50.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[29355]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:27:50.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[29355]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:27:50.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:50.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[29355]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:50.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:50.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[40592]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[40592]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:27:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[40592]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:27:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[40592]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:50.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:50.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:50.930 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2[56903]: 2024-09-11T08:27:50.655+0000 7f5d034a7640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T08:27:50.930 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:50 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2[56903]: 2024-09-11T08:27:50.659+0000 7f5cfd2dc640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:27:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:50 smithi172 ceph-mon[32987]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:50 smithi172 ceph-mon[32987]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:27:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:50 smithi172 ceph-mon[32987]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:27:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:50 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:50 smithi172 ceph-mon[32987]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:27:50.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:50 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:50.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:50 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:51 smithi043 ceph-mon[29355]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:51 smithi043 ceph-mon[29355]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:27:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:51.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:51 smithi043 ceph-mon[40592]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:51.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:51 smithi043 ceph-mon[40592]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:27:51.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:51.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:51 smithi172 ceph-mon[32987]: from='osd.2 v2:172.21.15.43:6809/319507648' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:27:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:51 smithi172 ceph-mon[32987]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:27:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:27:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:27:52.578 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-09-11T08:27:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-09-11T08:27:52.578 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00952584 s, 1.1 GB/s 2024-09-11T08:27:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-09-11T08:27:52.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[40592]: purged_snaps scrub starts 2024-09-11T08:27:52.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[40592]: purged_snaps scrub ok 2024-09-11T08:27:52.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[40592]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:52.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[40592]: osd.2 v2:172.21.15.43:6809/319507648 boot 2024-09-11T08:27:52.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[40592]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:27:52.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:52.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[29355]: purged_snaps scrub starts 2024-09-11T08:27:52.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[29355]: purged_snaps scrub ok 2024-09-11T08:27:52.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[29355]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:52.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[29355]: osd.2 v2:172.21.15.43:6809/319507648 boot 2024-09-11T08:27:52.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[29355]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:27:52.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:52 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:52 smithi172 ceph-mon[32987]: purged_snaps scrub starts 2024-09-11T08:27:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:52 smithi172 ceph-mon[32987]: purged_snaps scrub ok 2024-09-11T08:27:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:52 smithi172 ceph-mon[32987]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:27:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:52 smithi172 ceph-mon[32987]: osd.2 v2:172.21.15.43:6809/319507648 boot 2024-09-11T08:27:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:52 smithi172 ceph-mon[32987]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:27:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:52 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:27:53.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-09-11T08:27:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:27:54.097 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:53 smithi043 ceph-mon[40592]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:27:54.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:53 smithi043 ceph-mon[29355]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:27:54.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:53 smithi172 ceph-mon[32987]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:27:54.935 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[40592]: pgmap v76: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:54.935 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:27:54.935 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:27:54.935 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[40592]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:27:54.935 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:27:54.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[29355]: pgmap v76: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:54.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:27:54.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:27:54.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[29355]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:27:54.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:54 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:27:55.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:54 smithi172 ceph-mon[32987]: pgmap v76: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:55.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:54 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:27:55.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:54 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:27:55.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:54 smithi172 ceph-mon[32987]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:27:55.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:54 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:27:55.615 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:55 smithi043 sudo[61196]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:27:55.616 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:55 smithi043 sudo[61196]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:55.616 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:55 smithi043 sudo[61196]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:55.616 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:55 smithi043 sudo[61196]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 -- 172.21.15.43:0/3582210169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 msgr2=0x7f4a90104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 --2- 172.21.15.43:0/3582210169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 0x7f4a90104ce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009cd0 tx=0x7f4a88004d10 comp rx=0 tx=0).stop 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 -- 172.21.15.43:0/3582210169 shutdown_connections 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 --2- 172.21.15.43:0/3582210169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 0x7f4a90104ce0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 --2- 172.21.15.43:0/3582210169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a900fda90 0x7f4a900fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 --2- 172.21.15.43:0/3582210169 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a900fc7c0 0x7f4a900fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 -- 172.21.15.43:0/3582210169 >> 172.21.15.43:0/3582210169 conn(0x7f4a900f7f80 msgr2=0x7f4a900fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 -- 172.21.15.43:0/3582210169 shutdown_connections 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 -- 172.21.15.43:0/3582210169 wait complete. 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 Processor -- start 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.840+0000 7f4a9827a640 1 -- start start 2024-09-11T08:27:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a9827a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a900fc7c0 0x7f4a9010cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a9827a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a900fda90 0x7f4a9010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a9827a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 0x7f4a9010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a9827a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9010e5d0 con 0x7f4a900fe440 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a9827a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9010e740 con 0x7f4a900fc7c0 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a9827a640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9010e900 con 0x7f4a900fda90 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a900fda90 0x7f4a9010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a96817640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 0x7f4a9010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a900fda90 0x7f4a9010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:39336/0 (socket says 172.21.15.43:39336) 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 -- 172.21.15.43:0/1397482422 learned_addr learned my addr 172.21.15.43:0/1397482422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a96016640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a900fc7c0 0x7f4a9010cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 -- 172.21.15.43:0/1397482422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a900fc7c0 msgr2=0x7f4a9010cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a900fc7c0 0x7f4a9010cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 -- 172.21.15.43:0/1397482422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 msgr2=0x7f4a9010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 0x7f4a9010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 -- 172.21.15.43:0/1397482422 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c005040 con 0x7f4a900fda90 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a96817640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 0x7f4a9010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a96016640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a900fc7c0 0x7f4a9010cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a95815640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a900fda90 0x7f4a9010d0c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a84002ad0 tx=0x7f4a84004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.841+0000 7f4a82ffd640 1 -- 172.21.15.43:0/1397482422 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84007c30 con 0x7f4a900fda90 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.842+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a880098b0 con 0x7f4a900fda90 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.842+0000 7f4a82ffd640 1 -- 172.21.15.43:0/1397482422 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f4a8400f040 con 0x7f4a900fda90 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.842+0000 7f4a82ffd640 1 -- 172.21.15.43:0/1397482422 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84009340 con 0x7f4a900fda90 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.842+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a90115200 con 0x7f4a900fda90 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.843+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9005ed70 con 0x7f4a900fda90 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.843+0000 7f4a82ffd640 1 -- 172.21.15.43:0/1397482422 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 14) v1 ==== 90689+0+0 (secure 0 0 0) 0x7f4a840096f0 con 0x7f4a900fda90 2024-09-11T08:27:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.843+0000 7f4a82ffd640 1 --2- 172.21.15.43:0/1397482422 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4a7c06ed50 0x7f4a7c071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:27:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.844+0000 7f4a96016640 1 --2- 172.21.15.43:0/1397482422 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4a7c06ed50 0x7f4a7c071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:27:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.844+0000 7f4a82ffd640 1 -- 172.21.15.43:0/1397482422 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2507+0+0 (secure 0 0 0) 0x7f4a8408d380 con 0x7f4a900fda90 2024-09-11T08:27:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.844+0000 7f4a96016640 1 --2- 172.21.15.43:0/1397482422 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4a7c06ed50 0x7f4a7c071200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c007fd0 tx=0x7f4a8c009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:27:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:55.848+0000 7f4a82ffd640 1 -- 172.21.15.43:0/1397482422 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f4a8405b110 con 0x7f4a900fda90 2024-09-11T08:27:55.894 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:55 smithi043 sudo[61243]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:27:55.894 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:55 smithi043 sudo[61243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:55.894 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:55 smithi043 sudo[61243]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:55.894 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:27:55 smithi043 sudo[61243]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: Detected new or changed devices on smithi043 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: Detected new or changed devices on smithi043 2024-09-11T08:27:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:55.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:55.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:55 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:56.024+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a901075d0 con 0x7f4a7c06ed50 2024-09-11T08:27:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:27:56.077+0000 7f4a82ffd640 1 -- 172.21.15.43:0/1397482422 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7f4a84056d50 con 0x7f4a900fda90 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: Detected new or changed devices on smithi043 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:27:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:27:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:55 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:27:56.286 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:56 smithi043 sudo[61344]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:27:56.286 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:56 smithi043 sudo[61344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:56.286 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:56 smithi043 sudo[61344]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:56.286 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:56 smithi043 sudo[61344]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:56.859 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:56 smithi043 sudo[61427]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:27:56.859 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:56 smithi043 sudo[61427]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:56.859 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:56 smithi043 sudo[61427]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:56.859 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:27:56 smithi043 sudo[61427]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:56.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[29355]: osdmap e22: 3 total, 3 up, 3 in 2024-09-11T08:27:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[29355]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:57.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[40592]: osdmap e22: 3 total, 3 up, 3 in 2024-09-11T08:27:57.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[40592]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:57.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:57.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:57.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:57.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[40592]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:27:57.123 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:56 smithi043 ceph-mon[29355]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:27:57.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:56 smithi172 ceph-mon[32987]: osdmap e22: 3 total, 3 up, 3 in 2024-09-11T08:27:57.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:56 smithi172 ceph-mon[32987]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:27:57.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:56 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:27:57.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:56 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:27:57.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:56 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:27:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:56 smithi172 ceph-mon[32987]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:27:57.373 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:57 smithi043 sudo[61438]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:27:57.373 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:57 smithi043 sudo[61438]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:57.373 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:57 smithi043 sudo[61438]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:57.373 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:57 smithi043 sudo[61438]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:57.894 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:57 smithi043 sudo[61464]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:27:57.894 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:57 smithi043 sudo[61464]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:57.894 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:57 smithi043 sudo[61464]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:57.894 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:27:57 smithi043 sudo[61464]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:58.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:58 smithi043 sudo[61512]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:27:58.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:58 smithi043 sudo[61512]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:58 smithi043 sudo[61512]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:58 smithi043 sudo[61512]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:58.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:58 smithi043 sudo[61535]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:27:58.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:58 smithi043 sudo[61535]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:58.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:58 smithi043 sudo[61535]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:58.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:58 smithi043 sudo[61535]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:59.027 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:58 smithi172 sudo[35749]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:27:59.028 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:58 smithi172 sudo[35749]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:59.028 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:58 smithi172 sudo[35749]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:59.028 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:58 smithi172 sudo[35749]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:59.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:59.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:27:59.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4245838515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]: dispatch 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]: dispatch 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]': finished 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:27:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4245838515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]: dispatch 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]: dispatch 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]': finished 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:27:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:27:59.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:27:59.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 sudo[35752]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 sudo[35752]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 sudo[35752]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 sudo[35752]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4245838515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]: dispatch 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:27:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]: dispatch 2024-09-11T08:27:59.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:27:59.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:27:59.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f653df6d-614c-4e93-b9dc-3f164fba74d6"}]': finished 2024-09-11T08:27:59.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:27:59.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:27:59.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:27:59.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:27:59.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:27:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:27:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 sudo[61572]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:27:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 sudo[61572]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:27:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 sudo[61572]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:27:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 sudo[61572]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:28:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:28:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:28:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:28:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2228817576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 sudo[61575]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 sudo[61575]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 sudo[61575]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:27:59 smithi043 sudo[61575]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:28:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:28:00.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2228817576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:28:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:28:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:28:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:28:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:28:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:28:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:28:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2228817576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:02.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:02 smithi043 ceph-mon[29355]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:02.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:02 smithi043 ceph-mon[40592]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:02 smithi172 ceph-mon[32987]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:04.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:04 smithi043 ceph-mon[40592]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:04 smithi043 ceph-mon[29355]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:04 smithi172 ceph-mon[32987]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:06.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:06 smithi043 ceph-mon[40592]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T08:28:06.543 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:06 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:28:06.543 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:06 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:06.543 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:06 smithi043 ceph-mon[40592]: Deploying daemon osd.3 on smithi043 2024-09-11T08:28:06.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:06 smithi043 ceph-mon[29355]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T08:28:06.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:06 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:28:06.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:06 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:06.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:06 smithi043 ceph-mon[29355]: Deploying daemon osd.3 on smithi043 2024-09-11T08:28:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:06 smithi172 ceph-mon[32987]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T08:28:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:06 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:28:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:06 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:06 smithi172 ceph-mon[32987]: Deploying daemon osd.3 on smithi043 2024-09-11T08:28:08.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:08 smithi043 ceph-mon[40592]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T08:28:08.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:08 smithi043 ceph-mon[29355]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T08:28:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:08 smithi172 ceph-mon[32987]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T08:28:10.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:10 smithi043 ceph-mon[29355]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T08:28:10.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:10 smithi043 ceph-mon[40592]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T08:28:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:10 smithi172 ceph-mon[32987]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T08:28:11.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:11.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:11.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:11.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:11.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:11.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:12 smithi043 ceph-mon[40592]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T08:28:12.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:12 smithi043 ceph-mon[29355]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T08:28:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:12 smithi172 ceph-mon[32987]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T08:28:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.304+0000 7f4a82ffd640 1 -- 172.21.15.43:0/1397482422 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4a901075d0 con 0x7f4a7c06ed50 2024-09-11T08:28:14.307 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-09-11T08:28:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4a7c06ed50 msgr2=0x7f4a7c071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 --2- 172.21.15.43:0/1397482422 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4a7c06ed50 0x7f4a7c071200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c007fd0 tx=0x7f4a8c009da0 comp rx=0 tx=0).stop 2024-09-11T08:28:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a900fda90 msgr2=0x7f4a9010d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a900fda90 0x7f4a9010d0c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a84002ad0 tx=0x7f4a84004600 comp rx=0 tx=0).stop 2024-09-11T08:28:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 shutdown_connections 2024-09-11T08:28:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 --2- 172.21.15.43:0/1397482422 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4a7c06ed50 0x7f4a7c071200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a900fda90 0x7f4a9010d0c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:14.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900fe440 0x7f4a9010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:14.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 --2- 172.21.15.43:0/1397482422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a900fc7c0 0x7f4a9010cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:14.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 >> 172.21.15.43:0/1397482422 conn(0x7f4a900f7f80 msgr2=0x7f4a900fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:28:14.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 shutdown_connections 2024-09-11T08:28:14.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:28:14.307+0000 7f4a9827a640 1 -- 172.21.15.43:0/1397482422 wait complete. 2024-09-11T08:28:14.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.919 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T08:28:14.922 INFO:tasks.cephadm:Deploying osd.4 on smithi172 with /dev/vg_nvme/lv_4... 2024-09-11T08:28:14.922 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T08:28:14.950 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:14.951 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:15.175 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:28:15.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-mon[40592]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:28:15.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-mon[40592]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:28:15.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:15.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-mon[40592]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:28:15.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-mon[29355]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:28:15.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-mon[29355]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:28:15.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:15.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-mon[29355]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:28:15.852 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:28:15.634+0000 7f0f7547b640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T08:28:15.852 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:28:15 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:28:15.638+0000 7f0f6fab1640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:28:15.891 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:15 smithi172 ceph-mon[32987]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:28:15.891 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:15 smithi172 ceph-mon[32987]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:28:15.891 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:15 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:15.891 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:15 smithi172 ceph-mon[32987]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T08:28:16.720 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T08:28:16.802 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T08:28:16.803 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:16 smithi172 ceph-mon[32987]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:16.803 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:16 smithi172 ceph-mon[32987]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:28:16.803 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:16 smithi172 ceph-mon[32987]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T08:28:16.803 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:16 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:16.803 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:16 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:16.816 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-09-11T08:28:16.816 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-09-11T08:28:16.816 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114744 s, 914 MB/s 2024-09-11T08:28:16.816 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-09-11T08:28:16.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[29355]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[29355]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[29355]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[40592]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[40592]: from='osd.3 v2:172.21.15.43:6813/87557626' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[40592]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:16.855 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:16 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:17.447 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_4 2024-09-11T08:28:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:28:17.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:17 smithi172 ceph-mon[32987]: purged_snaps scrub starts 2024-09-11T08:28:17.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:17 smithi172 ceph-mon[32987]: purged_snaps scrub ok 2024-09-11T08:28:17.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:17 smithi172 ceph-mon[32987]: osd.3 v2:172.21.15.43:6813/87557626 boot 2024-09-11T08:28:17.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:17 smithi172 ceph-mon[32987]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:28:17.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:17 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[29355]: purged_snaps scrub starts 2024-09-11T08:28:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[29355]: purged_snaps scrub ok 2024-09-11T08:28:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[29355]: osd.3 v2:172.21.15.43:6813/87557626 boot 2024-09-11T08:28:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[29355]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:28:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[40592]: purged_snaps scrub starts 2024-09-11T08:28:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[40592]: purged_snaps scrub ok 2024-09-11T08:28:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[40592]: osd.3 v2:172.21.15.43:6813/87557626 boot 2024-09-11T08:28:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[40592]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:28:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:17 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:28:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:18 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:28:18.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T08:28:18.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:18 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:19.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 -- 172.21.15.172:0/3591618327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 msgr2=0x7fcb30104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:19.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 --2- 172.21.15.172:0/3591618327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 0x7fcb30104cd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb28004950 tx=0x7fcb28009da0 comp rx=0 tx=0).stop 2024-09-11T08:28:19.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 -- 172.21.15.172:0/3591618327 shutdown_connections 2024-09-11T08:28:19.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 --2- 172.21.15.172:0/3591618327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 0x7fcb30104cd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:19.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 --2- 172.21.15.172:0/3591618327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300fda80 0x7fcb300fdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:19.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 --2- 172.21.15.172:0/3591618327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb300fc7b0 0x7fcb300fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:19.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 -- 172.21.15.172:0/3591618327 >> 172.21.15.172:0/3591618327 conn(0x7fcb300f7f90 msgr2=0x7fcb300fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:28:19.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 -- 172.21.15.172:0/3591618327 shutdown_connections 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.323+0000 7fcb353e5640 1 -- 172.21.15.172:0/3591618327 wait complete. 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.324+0000 7fcb353e5640 1 Processor -- start 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.324+0000 7fcb353e5640 1 -- start start 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.324+0000 7fcb353e5640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb300fc7b0 0x7fcb3010cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.324+0000 7fcb353e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300fda80 0x7fcb3010d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.324+0000 7fcb353e5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 0x7fcb30114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.324+0000 7fcb353e5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30114650 con 0x7fcb300fda80 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.324+0000 7fcb353e5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301147c0 con 0x7fcb300fe430 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.324+0000 7fcb353e5640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30114980 con 0x7fcb300fc7b0 2024-09-11T08:28:19.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 0x7fcb30114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 0x7fcb30114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50500/0 (socket says 172.21.15.172:50500) 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 -- 172.21.15.172:0/4096551070 learned_addr learned my addr 172.21.15.172:0/4096551070 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2effd640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb300fc7b0 0x7fcb3010cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2e7fc640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300fda80 0x7fcb3010d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 -- 172.21.15.172:0/4096551070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb300fc7b0 msgr2=0x7fcb3010cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb300fc7b0 0x7fcb3010cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 -- 172.21.15.172:0/4096551070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300fda80 msgr2=0x7fcb3010d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300fda80 0x7fcb3010d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 -- 172.21.15.172:0/4096551070 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28005040 con 0x7fcb300fe430 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb2f7fe640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 0x7fcb30114080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800b020 tx=0x7fcb28009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb177fe640 1 -- 172.21.15.172:0/4096551070 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb280325e0 con 0x7fcb300fe430 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb177fe640 1 -- 172.21.15.172:0/4096551070 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fcb28032bb0 con 0x7fcb300fe430 2024-09-11T08:28:19.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb30114c00 con 0x7fcb300fe430 2024-09-11T08:28:19.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb177fe640 1 -- 172.21.15.172:0/4096551070 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2803c9c0 con 0x7fcb300fe430 2024-09-11T08:28:19.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.325+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb30115120 con 0x7fcb300fe430 2024-09-11T08:28:19.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.326+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3004bf60 con 0x7fcb300fe430 2024-09-11T08:28:19.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.327+0000 7fcb177fe640 1 -- 172.21.15.172:0/4096551070 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7fcb28030020 con 0x7fcb300fe430 2024-09-11T08:28:19.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.327+0000 7fcb177fe640 1 --2- 172.21.15.172:0/4096551070 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb1806eec0 0x7fcb18071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:28:19.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.327+0000 7fcb177fe640 1 -- 172.21.15.172:0/4096551070 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 2799+0+0 (secure 0 0 0) 0x7fcb280ad620 con 0x7fcb300fe430 2024-09-11T08:28:19.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.327+0000 7fcb2effd640 1 --2- 172.21.15.172:0/4096551070 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb1806eec0 0x7fcb18071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:28:19.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.328+0000 7fcb2effd640 1 --2- 172.21.15.172:0/4096551070 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb1806eec0 0x7fcb18071370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb20007580 tx=0x7fcb20009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:28:19.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.332+0000 7fcb177fe640 1 -- 172.21.15.172:0/4096551070 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcb2807b3c0 con 0x7fcb300fe430 2024-09-11T08:28:19.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:19.495+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb30107800 con 0x7fcb1806eec0 2024-09-11T08:28:19.901 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:19 smithi172 ceph-mon[32987]: Detected new or changed devices on smithi043 2024-09-11T08:28:19.901 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:19 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-11T08:28:19.901 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:19 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:28:19.901 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:19 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:28:19.901 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:19 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[40592]: Detected new or changed devices on smithi043 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[29355]: Detected new or changed devices on smithi043 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:28:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:20.879 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:20 smithi172 ceph-mon[32987]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:20.879 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:20 smithi172 ceph-mon[32987]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:28:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:20 smithi043 ceph-mon[40592]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:20 smithi043 ceph-mon[40592]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:28:21.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:20 smithi043 ceph-mon[29355]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:21.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:20 smithi043 ceph-mon[29355]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:28:22.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:22 smithi172 ceph-mon[32987]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:22.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/3935521035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]: dispatch 2024-09-11T08:28:22.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]: dispatch 2024-09-11T08:28:22.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]': finished 2024-09-11T08:28:22.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:22 smithi172 ceph-mon[32987]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:28:22.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:22.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/1364001133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[29355]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/3935521035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]: dispatch 2024-09-11T08:28:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]: dispatch 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]': finished 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[29355]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/1364001133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[40592]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/3935521035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]: dispatch 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]: dispatch 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3083c1a6-6c94-47ed-bb13-86690fcebd11"}]': finished 2024-09-11T08:28:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[40592]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:28:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/1364001133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:24 smithi172 ceph-mon[32987]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:24 smithi043 ceph-mon[29355]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:24 smithi043 ceph-mon[40592]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:26.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:26 smithi172 ceph-mon[32987]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:27.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:26 smithi043 ceph-mon[29355]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:26 smithi043 ceph-mon[40592]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:28 smithi172 ceph-mon[32987]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:28:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:28 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:29.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:28 smithi043 ceph-mon[29355]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:29.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:28:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:28 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:29.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:28 smithi043 ceph-mon[40592]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:29.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:28 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:28:29.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:28 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:29 smithi172 ceph-mon[32987]: Deploying daemon osd.4 on smithi172 2024-09-11T08:28:30.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:29 smithi043 ceph-mon[29355]: Deploying daemon osd.4 on smithi172 2024-09-11T08:28:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:29 smithi043 ceph-mon[40592]: Deploying daemon osd.4 on smithi172 2024-09-11T08:28:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:30 smithi172 ceph-mon[32987]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:31.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:30 smithi043 ceph-mon[29355]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:30 smithi043 ceph-mon[40592]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:32.959 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:32 smithi172 ceph-mon[32987]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:32.960 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:32.960 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:32.960 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:32 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:32 smithi043 ceph-mon[40592]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:33.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:32 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:32 smithi043 ceph-mon[29355]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:32 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:34.937 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:34 smithi172 ceph-mon[32987]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:34.947 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:34 smithi043 ceph-mon[40592]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:34.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:34 smithi043 ceph-mon[29355]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:35.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.648+0000 7fcb177fe640 1 -- 172.21.15.172:0/4096551070 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb30107800 con 0x7fcb1806eec0 2024-09-11T08:28:35.649 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 4 on host 'smithi172' 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.650+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb1806eec0 msgr2=0x7fcb18071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.650+0000 7fcb353e5640 1 --2- 172.21.15.172:0/4096551070 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb1806eec0 0x7fcb18071370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb20007580 tx=0x7fcb20009780 comp rx=0 tx=0).stop 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.650+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 msgr2=0x7fcb30114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.650+0000 7fcb353e5640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 0x7fcb30114080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800b020 tx=0x7fcb28009da0 comp rx=0 tx=0).stop 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.651+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 shutdown_connections 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.651+0000 7fcb353e5640 1 --2- 172.21.15.172:0/4096551070 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb1806eec0 0x7fcb18071370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.651+0000 7fcb353e5640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb300fe430 0x7fcb30114080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.651+0000 7fcb353e5640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300fda80 0x7fcb3010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:35.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.651+0000 7fcb353e5640 1 --2- 172.21.15.172:0/4096551070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb300fc7b0 0x7fcb3010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:35.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.651+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 >> 172.21.15.172:0/4096551070 conn(0x7fcb300f7f90 msgr2=0x7fcb300f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:28:35.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.651+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 shutdown_connections 2024-09-11T08:28:35.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:35.651+0000 7fcb353e5640 1 -- 172.21.15.172:0/4096551070 wait complete. 2024-09-11T08:28:36.261 DEBUG:teuthology.orchestra.run.smithi172:osd.4> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T08:28:36.263 INFO:tasks.cephadm:Deploying osd.5 on smithi172 with /dev/vg_nvme/lv_3... 2024-09-11T08:28:36.263 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T08:28:36.563 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:28:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[40592]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[40592]: from='osd.4 v2:172.21.15.172:6800/2140228578' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[40592]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[29355]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[29355]: from='osd.4 v2:172.21.15.172:6800/2140228578' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[29355]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:28:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:36 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.977 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:36 smithi172 ceph-mon[32987]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:36.977 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:36 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.978 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:36 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.978 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:36 smithi172 ceph-mon[32987]: from='osd.4 v2:172.21.15.172:6800/2140228578' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:28:36.978 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:36 smithi172 ceph-mon[32987]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:28:36.978 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:36 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:36.978 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:36 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:37.692 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:37 smithi172 ceph-mon[32987]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:28:37.692 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:37 smithi172 ceph-mon[32987]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:28:37.692 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:37 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:37.692 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:37 smithi172 ceph-mon[32987]: from='osd.4 v2:172.21.15.172:6800/2140228578' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:28:37.692 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:37 smithi172 ceph-mon[32987]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:28:37.693 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:28:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:28:37.685+0000 7fa62a068640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T08:28:37.693 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:28:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:28:37.690+0000 7fa623e9d640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:28:38.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[40592]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:28:38.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[40592]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:28:38.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:38.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[40592]: from='osd.4 v2:172.21.15.172:6800/2140228578' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:28:38.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[40592]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:28:38.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[29355]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:28:38.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[29355]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:28:38.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:38.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[29355]: from='osd.4 v2:172.21.15.172:6800/2140228578' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:28:38.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:37 smithi043 ceph-mon[29355]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:28:38.788 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:38 smithi172 ceph-mon[32987]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:38.788 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:38 smithi172 ceph-mon[32987]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:28:38.788 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:38 smithi172 ceph-mon[32987]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T08:28:38.789 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:38.789 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:39.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[29355]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:39.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[29355]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:28:39.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[29355]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T08:28:39.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:39.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:39.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[40592]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:28:39.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[40592]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:28:39.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[40592]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T08:28:39.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:39.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:39.371 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T08:28:39.468 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T08:28:39.485 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-09-11T08:28:39.485 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-09-11T08:28:39.485 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140156 s, 748 MB/s 2024-09-11T08:28:39.485 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-09-11T08:28:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:39 smithi172 ceph-mon[32987]: purged_snaps scrub starts 2024-09-11T08:28:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:39 smithi172 ceph-mon[32987]: purged_snaps scrub ok 2024-09-11T08:28:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:39 smithi172 ceph-mon[32987]: osd.4 v2:172.21.15.172:6800/2140228578 boot 2024-09-11T08:28:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:39 smithi172 ceph-mon[32987]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:28:39.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:39 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:39.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:39 smithi172 ceph-mon[32987]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:28:40.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[29355]: purged_snaps scrub starts 2024-09-11T08:28:40.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[29355]: purged_snaps scrub ok 2024-09-11T08:28:40.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[29355]: osd.4 v2:172.21.15.172:6800/2140228578 boot 2024-09-11T08:28:40.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[29355]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:28:40.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:40.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[29355]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:28:40.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[40592]: purged_snaps scrub starts 2024-09-11T08:28:40.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[40592]: purged_snaps scrub ok 2024-09-11T08:28:40.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[40592]: osd.4 v2:172.21.15.172:6800/2140228578 boot 2024-09-11T08:28:40.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[40592]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:28:40.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:28:40.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:39 smithi043 ceph-mon[40592]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:28:40.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_3 2024-09-11T08:28:40.794 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:40 smithi172 ceph-mon[32987]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:40.795 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:40 smithi172 ceph-mon[32987]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:28:40.863 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:28:41.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:40 smithi043 ceph-mon[40592]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:41.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:40 smithi043 ceph-mon[40592]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:28:41.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:40 smithi043 ceph-mon[29355]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:41.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:40 smithi043 ceph-mon[29355]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:28:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: Detected new or changed devices on smithi172 2024-09-11T08:28:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:41.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:41 smithi172 ceph-mon[32987]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:42.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: Detected new or changed devices on smithi172 2024-09-11T08:28:42.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[40592]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:42.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: Detected new or changed devices on smithi172 2024-09-11T08:28:42.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:28:42.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-11T08:28:42.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:42.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:42.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:42.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:42.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:42.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:42.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:41 smithi043 ceph-mon[29355]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.450+0000 7f803886e640 1 -- 172.21.15.172:0/3520516004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 msgr2=0x7f80340fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.450+0000 7f803886e640 1 --2- 172.21.15.172:0/3520516004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 0x7f80340fbaf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f802c004950 tx=0x7f802c009da0 comp rx=0 tx=0).stop 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 -- 172.21.15.172:0/3520516004 shutdown_connections 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 --2- 172.21.15.172:0/3520516004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80340fc910 0x7f80340fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 --2- 172.21.15.172:0/3520516004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80340fd2c0 0x7f8034103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 --2- 172.21.15.172:0/3520516004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 0x7f80340fbaf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 -- 172.21.15.172:0/3520516004 >> 172.21.15.172:0/3520516004 conn(0x7f80340f6ea0 msgr2=0x7f80340f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 -- 172.21.15.172:0/3520516004 shutdown_connections 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 -- 172.21.15.172:0/3520516004 wait complete. 2024-09-11T08:28:42.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 Processor -- start 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 -- start start 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 0x7f80340ff580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80340fc910 0x7f80340fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80340fd2c0 0x7f80340fe190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340fe8d0 con 0x7f80340fc910 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340fea40 con 0x7f80340fb6e0 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803886e640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8034116ec0 con 0x7f80340fd2c0 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 0x7f80340ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 0x7f80340ff580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58586/0 (socket says 172.21.15.172:58586) 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.451+0000 7f803259c640 1 -- 172.21.15.172:0/3066449927 learned_addr learned my addr 172.21.15.172:0/3066449927 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f8032d9d640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80340fd2c0 0x7f80340fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:28:42.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f8031d9b640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80340fc910 0x7f80340fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f803259c640 1 -- 172.21.15.172:0/3066449927 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80340fd2c0 msgr2=0x7f80340fe190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f803259c640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80340fd2c0 0x7f80340fe190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f803259c640 1 -- 172.21.15.172:0/3066449927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80340fc910 msgr2=0x7f80340fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f803259c640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80340fc910 0x7f80340fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f803259c640 1 -- 172.21.15.172:0/3066449927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c005040 con 0x7f80340fb6e0 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f803259c640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 0x7f80340ff580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f802c02d040 tx=0x7f802c035040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f802affd640 1 -- 172.21.15.172:0/3066449927 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c02abf0 con 0x7f80340fb6e0 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f802affd640 1 -- 172.21.15.172:0/3066449927 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f802c02ad50 con 0x7f80340fb6e0 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80341170c0 con 0x7f80340fb6e0 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f802affd640 1 -- 172.21.15.172:0/3066449927 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c033690 con 0x7f80340fb6e0 2024-09-11T08:28:42.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.452+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8034117550 con 0x7f80340fb6e0 2024-09-11T08:28:42.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.453+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803404bf60 con 0x7f80340fb6e0 2024-09-11T08:28:42.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.454+0000 7f802affd640 1 -- 172.21.15.172:0/3066449927 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7f802c030020 con 0x7f80340fb6e0 2024-09-11T08:28:42.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.454+0000 7f802affd640 1 --2- 172.21.15.172:0/3066449927 >> v2:172.21.15.43:6800/1424089977 conn(0x7f802006ee90 0x7f8020071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:28:42.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.455+0000 7f802affd640 1 -- 172.21.15.172:0/3066449927 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3201+0+0 (secure 0 0 0) 0x7f802c0ade90 con 0x7f80340fb6e0 2024-09-11T08:28:42.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.455+0000 7f8031d9b640 1 --2- 172.21.15.172:0/3066449927 >> v2:172.21.15.43:6800/1424089977 conn(0x7f802006ee90 0x7f8020071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:28:42.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.455+0000 7f8031d9b640 1 --2- 172.21.15.172:0/3066449927 >> v2:172.21.15.43:6800/1424089977 conn(0x7f802006ee90 0x7f8020071340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f801c006fd0 tx=0x7f801c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:28:42.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.459+0000 7f802affd640 1 -- 172.21.15.172:0/3066449927 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f802c07baa0 con 0x7f80340fb6e0 2024-09-11T08:28:42.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:42.620+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f80341066d0 con 0x7f802006ee90 2024-09-11T08:28:43.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:42 smithi172 ceph-mon[32987]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:28:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:42 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:28:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:42 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:28:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:42 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:43.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:42 smithi043 ceph-mon[29355]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:28:43.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:42 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:28:43.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:42 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:28:43.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:42 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:42 smithi043 ceph-mon[40592]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:28:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:42 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:28:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:42 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:28:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:42 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:44.001 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:43 smithi172 ceph-mon[32987]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:44.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:43 smithi043 ceph-mon[29355]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:44.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:43 smithi043 ceph-mon[40592]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:45.184 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/3217862956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]: dispatch 2024-09-11T08:28:45.185 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]: dispatch 2024-09-11T08:28:45.185 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]': finished 2024-09-11T08:28:45.185 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:44 smithi172 ceph-mon[32987]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:28:45.185 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:44 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:28:45.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/3217862956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]: dispatch 2024-09-11T08:28:45.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]: dispatch 2024-09-11T08:28:45.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]': finished 2024-09-11T08:28:45.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[29355]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:28:45.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:28:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/3217862956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]: dispatch 2024-09-11T08:28:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]: dispatch 2024-09-11T08:28:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dab026a9-396f-4828-8d32-9c7e6142d1b2"}]': finished 2024-09-11T08:28:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[40592]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:28:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:44 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:28:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:45 smithi172 ceph-mon[32987]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T08:28:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/1078909733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:46.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:45 smithi043 ceph-mon[29355]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T08:28:46.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/1078909733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:46.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:45 smithi043 ceph-mon[40592]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T08:28:46.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/1078909733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:28:48.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:48 smithi043 ceph-mon[29355]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:28:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:48 smithi043 ceph-mon[40592]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:28:48.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:48 smithi172 ceph-mon[32987]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:28:50.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:50 smithi043 ceph-mon[29355]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T08:28:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:50 smithi043 ceph-mon[40592]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T08:28:50.698 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:50 smithi172 ceph-mon[32987]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T08:28:51.642 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:28:51.642 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:28:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:28:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:52.642 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:52 smithi172 ceph-mon[32987]: Deploying daemon osd.5 on smithi172 2024-09-11T08:28:52.642 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:52 smithi172 ceph-mon[32987]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:28:52.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:52 smithi043 ceph-mon[29355]: Deploying daemon osd.5 on smithi172 2024-09-11T08:28:52.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:52 smithi043 ceph-mon[29355]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:28:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:52 smithi043 ceph-mon[40592]: Deploying daemon osd.5 on smithi172 2024-09-11T08:28:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:52 smithi043 ceph-mon[40592]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:28:54.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:54 smithi043 ceph-mon[29355]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:28:54.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:54 smithi043 ceph-mon[40592]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:28:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:54 smithi172 ceph-mon[32987]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:28:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:56 smithi172 ceph-mon[32987]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:56 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:56 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:56 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:56.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:56 smithi043 ceph-mon[29355]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:56.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:56.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:56.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:56 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:56 smithi043 ceph-mon[40592]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:56 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:56 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:56 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[40592]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[29355]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:58 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.663 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:58 smithi172 ceph-mon[32987]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:28:58.663 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:58 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:58 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:58.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:58 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:28:58.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:58 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:28:58.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:58 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:59.317 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 5 on host 'smithi172' 2024-09-11T08:28:59.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.317+0000 7f802affd640 1 -- 172.21.15.172:0/3066449927 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f80341066d0 con 0x7f802006ee90 2024-09-11T08:28:59.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 >> v2:172.21.15.43:6800/1424089977 conn(0x7f802006ee90 msgr2=0x7f8020071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:59.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 --2- 172.21.15.172:0/3066449927 >> v2:172.21.15.43:6800/1424089977 conn(0x7f802006ee90 0x7f8020071340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f801c006fd0 tx=0x7f801c008040 comp rx=0 tx=0).stop 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 msgr2=0x7f80340ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 0x7f80340ff580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f802c02d040 tx=0x7f802c035040 comp rx=0 tx=0).stop 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 shutdown_connections 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 --2- 172.21.15.172:0/3066449927 >> v2:172.21.15.43:6800/1424089977 conn(0x7f802006ee90 0x7f8020071340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80340fd2c0 0x7f80340fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80340fc910 0x7f80340fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 --2- 172.21.15.172:0/3066449927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80340fb6e0 0x7f80340ff580 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 >> 172.21.15.172:0/3066449927 conn(0x7f80340f6ea0 msgr2=0x7f80340f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 shutdown_connections 2024-09-11T08:28:59.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:28:59.319+0000 7f803886e640 1 -- 172.21.15.172:0/3066449927 wait complete. 2024-09-11T08:28:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[40592]: from='osd.5 v2:172.21.15.172:6804/2848105378' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:28:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[40592]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:28:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[29355]: from='osd.5 v2:172.21.15.172:6804/2848105378' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:28:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[29355]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:28:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:28:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:59.675 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:59 smithi172 ceph-mon[32987]: from='osd.5 v2:172.21.15.172:6804/2848105378' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:28:59.675 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:59 smithi172 ceph-mon[32987]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:28:59.675 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:28:59.675 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:59.675 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:28:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:28:59.957 DEBUG:teuthology.orchestra.run.smithi172:osd.5> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T08:28:59.959 INFO:tasks.cephadm:Deploying osd.6 on smithi172 with /dev/vg_nvme/lv_2... 2024-09-11T08:28:59.959 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T08:29:00.520 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:00 smithi172 ceph-mon[32987]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:29:00.521 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:00 smithi172 ceph-mon[32987]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:29:00.521 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:00 smithi172 ceph-mon[32987]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:29:00.521 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:00 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:00.521 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:00 smithi172 ceph-mon[32987]: from='osd.5 v2:172.21.15.172:6804/2848105378' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:00.521 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:00 smithi172 ceph-mon[32987]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:00.521 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:29:00 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:29:00.406+0000 7f5dacc22640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T08:29:00.521 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:29:00 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:29:00.411+0000 7f5da9a5d640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:29:00.735 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:29:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[29355]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:29:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[29355]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:29:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[29355]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:29:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[29355]: from='osd.5 v2:172.21.15.172:6804/2848105378' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[29355]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[40592]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:29:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[40592]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:29:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[40592]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:29:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[40592]: from='osd.5 v2:172.21.15.172:6804/2848105378' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:00 smithi043 ceph-mon[40592]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:01 smithi172 ceph-mon[32987]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:01 smithi172 ceph-mon[32987]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:29:01.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:01 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:01.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:01 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:01 smithi043 ceph-mon[40592]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:01 smithi043 ceph-mon[40592]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:29:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:01 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:01.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:01 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:01 smithi043 ceph-mon[29355]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:01 smithi043 ceph-mon[29355]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:29:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:01 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:02.668 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:02 smithi172 ceph-mon[32987]: purged_snaps scrub starts 2024-09-11T08:29:02.668 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:02 smithi172 ceph-mon[32987]: purged_snaps scrub ok 2024-09-11T08:29:02.668 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:02 smithi172 ceph-mon[32987]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:29:02.668 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:02.668 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:02 smithi172 ceph-mon[32987]: osd.5 v2:172.21.15.172:6804/2848105378 boot 2024-09-11T08:29:02.668 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:02 smithi172 ceph-mon[32987]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:29:02.669 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[40592]: purged_snaps scrub starts 2024-09-11T08:29:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[40592]: purged_snaps scrub ok 2024-09-11T08:29:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[40592]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:29:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[40592]: osd.5 v2:172.21.15.172:6804/2848105378 boot 2024-09-11T08:29:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[40592]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:29:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[29355]: purged_snaps scrub starts 2024-09-11T08:29:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[29355]: purged_snaps scrub ok 2024-09-11T08:29:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[29355]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:29:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[29355]: osd.5 v2:172.21.15.172:6804/2848105378 boot 2024-09-11T08:29:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[29355]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:29:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:29:03.243 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T08:29:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T08:29:03.358 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-09-11T08:29:03.358 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-09-11T08:29:03.358 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0146108 s, 718 MB/s 2024-09-11T08:29:03.358 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-09-11T08:29:03.690 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:03 smithi172 ceph-mon[32987]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:29:03.690 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:03 smithi172 ceph-mon[32987]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:29:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:03 smithi043 ceph-mon[40592]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:29:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:03 smithi043 ceph-mon[40592]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:29:03.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:03 smithi043 ceph-mon[29355]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:29:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:03 smithi043 ceph-mon[29355]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:29:04.016 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_2 2024-09-11T08:29:04.552 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:04 smithi172 ceph-mon[32987]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:29:04.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:04 smithi043 ceph-mon[29355]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:04.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:04 smithi043 ceph-mon[40592]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:05.878 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: Detected new or changed devices on smithi172 2024-09-11T08:29:05.878 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.878 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.878 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:05.879 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:05.879 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-11T08:29:05.879 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.879 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:05.879 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:05.879 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: Detected new or changed devices on smithi172 2024-09-11T08:29:05.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-11T08:29:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: Detected new or changed devices on smithi172 2024-09-11T08:29:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:05.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-11T08:29:05.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:05.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:05.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:05.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:06.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 -- 172.21.15.172:0/3755526868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fdaa0 msgr2=0x7fcb980fdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:06.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/3755526868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fdaa0 0x7fcb980fdf10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcb94004950 tx=0x7fcb94009da0 comp rx=0 tx=0).stop 2024-09-11T08:29:06.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 -- 172.21.15.172:0/3755526868 shutdown_connections 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/3755526868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb980fe450 0x7fcb98104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/3755526868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fdaa0 0x7fcb980fdf10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/3755526868 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb980fc7d0 0x7fcb980fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 -- 172.21.15.172:0/3755526868 >> 172.21.15.172:0/3755526868 conn(0x7fcb980f7f90 msgr2=0x7fcb980fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 -- 172.21.15.172:0/3755526868 shutdown_connections 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 -- 172.21.15.172:0/3755526868 wait complete. 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 Processor -- start 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.214+0000 7fcb9fd53640 1 -- start start 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9fd53640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fc7d0 0x7fcb9810cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9fd53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb980fdaa0 0x7fcb9810d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9fd53640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb980fe450 0x7fcb98114390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9fd53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98114960 con 0x7fcb980fdaa0 2024-09-11T08:29:06.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9fd53640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98114ad0 con 0x7fcb980fc7d0 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9fd53640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98114c90 con 0x7fcb980fe450 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fc7d0 0x7fcb9810cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fc7d0 0x7fcb9810cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56832/0 (socket says 172.21.15.172:56832) 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 -- 172.21.15.172:0/1503569704 learned_addr learned my addr 172.21.15.172:0/1503569704 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9d2ee640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb980fdaa0 0x7fcb9810d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9e2f0640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb980fe450 0x7fcb98114390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 -- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb980fe450 msgr2=0x7fcb98114390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb980fe450 0x7fcb98114390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 -- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb980fdaa0 msgr2=0x7fcb9810d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb980fdaa0 0x7fcb9810d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 -- 172.21.15.172:0/1503569704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94005040 con 0x7fcb980fc7d0 2024-09-11T08:29:06.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9e2f0640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb980fe450 0x7fcb98114390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9daef640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fc7d0 0x7fcb9810cdf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00b810 tx=0x7fcb8c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb8a7fc640 1 -- 172.21.15.172:0/1503569704 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00be60 con 0x7fcb980fc7d0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb8a7fc640 1 -- 172.21.15.172:0/1503569704 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fcb8c00cba0 con 0x7fcb980fc7d0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb98114f70 con 0x7fcb980fc7d0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb8a7fc640 1 -- 172.21.15.172:0/1503569704 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c015b60 con 0x7fcb980fc7d0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.215+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb98115300 con 0x7fcb980fc7d0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.216+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9804bf60 con 0x7fcb980fc7d0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.217+0000 7fcb8a7fc640 1 -- 172.21.15.172:0/1503569704 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7fcb8c012070 con 0x7fcb980fc7d0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.217+0000 7fcb8a7fc640 1 --2- 172.21.15.172:0/1503569704 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb8406eec0 0x7fcb84071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.217+0000 7fcb8a7fc640 1 -- 172.21.15.172:0/1503569704 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 3493+0+0 (secure 0 0 0) 0x7fcb8c08d0a0 con 0x7fcb980fc7d0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.217+0000 7fcb9d2ee640 1 --2- 172.21.15.172:0/1503569704 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb8406eec0 0x7fcb84071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.218+0000 7fcb9d2ee640 1 --2- 172.21.15.172:0/1503569704 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb8406eec0 0x7fcb84071370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb940094b0 tx=0x7fcb94035040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.222+0000 7fcb8a7fc640 1 -- 172.21.15.172:0/1503569704 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcb8c05a970 con 0x7fcb980fc7d0 2024-09-11T08:29:06.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:06.386+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb98107810 con 0x7fcb8406eec0 2024-09-11T08:29:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:06 smithi172 ceph-mon[32987]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:06 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:29:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:06 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:29:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:06 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:06 smithi043 ceph-mon[40592]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:06 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:29:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:06 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:29:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:06 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:06 smithi043 ceph-mon[29355]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:06 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:29:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:06 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:29:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:06 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:07.795 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:07 smithi172 ceph-mon[32987]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:07 smithi043 ceph-mon[40592]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:07.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:07 smithi043 ceph-mon[29355]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:08.611 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:08 smithi043 ceph-mon[40592]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:08.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:08 smithi043 ceph-mon[29355]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:08 smithi172 ceph-mon[32987]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/3102456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]: dispatch 2024-09-11T08:29:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]: dispatch 2024-09-11T08:29:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]': finished 2024-09-11T08:29:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[29355]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:29:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/2816347820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:29:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/3102456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]: dispatch 2024-09-11T08:29:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]: dispatch 2024-09-11T08:29:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]': finished 2024-09-11T08:29:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[40592]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:29:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/2816347820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:29:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/3102456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]: dispatch 2024-09-11T08:29:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]: dispatch 2024-09-11T08:29:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06710545-d1ff-4929-8dbe-97868b7dea5e"}]': finished 2024-09-11T08:29:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:09 smithi172 ceph-mon[32987]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:29:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:09 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/2816347820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:29:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:10 smithi172 ceph-mon[32987]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:10 smithi043 ceph-mon[40592]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:11.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:10 smithi043 ceph-mon[29355]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:12.914 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:12 smithi172 ceph-mon[32987]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:12 smithi043 ceph-mon[40592]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:13.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:12 smithi043 ceph-mon[29355]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:14 smithi172 ceph-mon[32987]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:29:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:14 smithi043 ceph-mon[40592]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:29:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:15.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:14 smithi043 ceph-mon[29355]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:29:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:15 smithi043 ceph-mon[29355]: Deploying daemon osd.6 on smithi172 2024-09-11T08:29:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:15 smithi043 ceph-mon[40592]: Deploying daemon osd.6 on smithi172 2024-09-11T08:29:15.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:15 smithi172 ceph-mon[32987]: Deploying daemon osd.6 on smithi172 2024-09-11T08:29:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:16 smithi172 ceph-mon[32987]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:17.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:16 smithi043 ceph-mon[29355]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:16 smithi043 ceph-mon[40592]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:18.936 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:18 smithi172 ceph-mon[32987]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:18 smithi043 ceph-mon[29355]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:19.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:18 smithi043 ceph-mon[40592]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:19.901 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:19 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:19.901 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:19 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:19.901 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:19 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:19 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:20.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:19 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:20.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:19 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:20.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:19 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:20 smithi043 ceph-mon[29355]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:20 smithi043 ceph-mon[40592]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:21.215 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:20 smithi172 ceph-mon[32987]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:22.948 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:22 smithi172 ceph-mon[32987]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:22.948 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:22.949 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:22.949 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:22.949 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:22.949 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:22.949 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:22 smithi172 ceph-mon[32987]: from='osd.6 v2:172.21.15.172:6808/2701192652' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:29:22.949 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:22 smithi172 ceph-mon[32987]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:29:22.986 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 6 on host 'smithi172' 2024-09-11T08:29:22.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.984+0000 7fcb8a7fc640 1 -- 172.21.15.172:0/1503569704 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb98107810 con 0x7fcb8406eec0 2024-09-11T08:29:22.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb8406eec0 msgr2=0x7fcb84071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:22.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/1503569704 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb8406eec0 0x7fcb84071370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb940094b0 tx=0x7fcb94035040 comp rx=0 tx=0).stop 2024-09-11T08:29:22.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fc7d0 msgr2=0x7fcb9810cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fc7d0 0x7fcb9810cdf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00b810 tx=0x7fcb8c0091d0 comp rx=0 tx=0).stop 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 shutdown_connections 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/1503569704 >> v2:172.21.15.43:6800/1424089977 conn(0x7fcb8406eec0 0x7fcb84071370 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb980fe450 0x7fcb98114390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb980fdaa0 0x7fcb9810d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 --2- 172.21.15.172:0/1503569704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb980fc7d0 0x7fcb9810cdf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 >> 172.21.15.172:0/1503569704 conn(0x7fcb980f7f90 msgr2=0x7fcb980f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 shutdown_connections 2024-09-11T08:29:22.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:22.987+0000 7fcb9fd53640 1 -- 172.21.15.172:0/1503569704 wait complete. 2024-09-11T08:29:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[29355]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[29355]: from='osd.6 v2:172.21.15.172:6808/2701192652' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[29355]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[40592]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[40592]: from='osd.6 v2:172.21.15.172:6808/2701192652' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:29:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:22 smithi043 ceph-mon[40592]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:29:23.634 DEBUG:teuthology.orchestra.run.smithi172:osd.6> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T08:29:23.636 INFO:tasks.cephadm:Deploying osd.7 on smithi172 with /dev/vg_nvme/lv_1... 2024-09-11T08:29:23.637 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T08:29:23.956 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:23 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:23.956 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:23 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:23.956 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:23 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:23.956 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:23 smithi172 ceph-mon[32987]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:29:23.956 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:23 smithi172 ceph-mon[32987]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:29:23.957 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:23 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:23.957 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:23 smithi172 ceph-mon[32987]: from='osd.6 v2:172.21.15.172:6808/2701192652' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:23.957 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:23 smithi172 ceph-mon[32987]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:24.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:24.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:24.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:24.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[29355]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[29355]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[29355]: from='osd.6 v2:172.21.15.172:6808/2701192652' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[29355]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[40592]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[40592]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:29:24.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:24.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[40592]: from='osd.6 v2:172.21.15.172:6808/2701192652' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:24.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:23 smithi043 ceph-mon[40592]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:24.405 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:29:24.639 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:29:24 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:29:24.287+0000 7f7fa904d640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T08:29:24.640 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:29:24 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:29:24.294+0000 7f7fa4685640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:29:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:24 smithi172 ceph-mon[32987]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:24 smithi172 ceph-mon[32987]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:24 smithi172 ceph-mon[32987]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T08:29:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:24 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:24 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[29355]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[29355]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[29355]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T08:29:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[40592]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:29:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[40592]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[40592]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T08:29:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:24 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:26.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[29355]: purged_snaps scrub starts 2024-09-11T08:29:26.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[29355]: purged_snaps scrub ok 2024-09-11T08:29:26.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[29355]: osd.6 v2:172.21.15.172:6808/2701192652 boot 2024-09-11T08:29:26.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[29355]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:29:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[29355]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[40592]: purged_snaps scrub starts 2024-09-11T08:29:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[40592]: purged_snaps scrub ok 2024-09-11T08:29:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[40592]: osd.6 v2:172.21.15.172:6808/2701192652 boot 2024-09-11T08:29:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[40592]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:29:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:26 smithi043 ceph-mon[40592]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:26.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:26 smithi172 ceph-mon[32987]: purged_snaps scrub starts 2024-09-11T08:29:26.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:26 smithi172 ceph-mon[32987]: purged_snaps scrub ok 2024-09-11T08:29:26.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:26 smithi172 ceph-mon[32987]: osd.6 v2:172.21.15.172:6808/2701192652 boot 2024-09-11T08:29:26.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:26 smithi172 ceph-mon[32987]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:29:26.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:26 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:29:26.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:26 smithi172 ceph-mon[32987]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:27.626 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:29:27.626 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:29:27.626 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.627 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.627 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.627 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.627 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.627 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.627 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:27.627 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:27.627 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:27 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:29:27.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:27.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:27.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:27.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:27 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:27.719 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:29:27.734 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-09-11T08:29:27.734 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-09-11T08:29:27.734 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131739 s, 796 MB/s 2024-09-11T08:29:27.734 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-09-11T08:29:28.334 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:28 smithi172 ceph-mon[32987]: Detected new or changed devices on smithi172 2024-09-11T08:29:28.334 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:28 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-11T08:29:28.334 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:28 smithi172 ceph-mon[32987]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T08:29:28.334 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:28 smithi172 ceph-mon[32987]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:28.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_1 2024-09-11T08:29:28.601 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:29:28.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:28 smithi043 ceph-mon[29355]: Detected new or changed devices on smithi172 2024-09-11T08:29:28.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:28 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-11T08:29:28.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:28 smithi043 ceph-mon[29355]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T08:29:28.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:28 smithi043 ceph-mon[29355]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:28.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:28 smithi043 ceph-mon[40592]: Detected new or changed devices on smithi172 2024-09-11T08:29:28.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:28 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-11T08:29:28.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:28 smithi043 ceph-mon[40592]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T08:29:28.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:28 smithi043 ceph-mon[40592]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3637503244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 msgr2=0x7f8f38103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3637503244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 0x7f8f38103be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009cd0 tx=0x7f8f30004d10 comp rx=0 tx=0).stop 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3637503244 shutdown_connections 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3637503244 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f380fc990 0x7f8f380fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3637503244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 0x7f8f38103be0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3637503244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f380fb6c0 0x7f8f380fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3637503244 >> 172.21.15.172:0/3637503244 conn(0x7f8f380f6ea0 msgr2=0x7f8f380f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3637503244 shutdown_connections 2024-09-11T08:29:30.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3637503244 wait complete. 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.271+0000 7f8f3dbea640 1 Processor -- start 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f3dbea640 1 -- start start 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f3dbea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f380fb6c0 0x7f8f3810b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f3dbea640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f380fc990 0x7f8f3810be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f3dbea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 0x7f8f38112e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f3dbea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38113460 con 0x7f8f380fb6c0 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f3dbea640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381135d0 con 0x7f8f380fd340 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f3dbea640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38113790 con 0x7f8f380fc990 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 0x7f8f38112e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 0x7f8f38112e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43860/0 (socket says 172.21.15.172:43860) 2024-09-11T08:29:30.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 -- 172.21.15.172:0/3271141787 learned_addr learned my addr 172.21.15.172:0/3271141787 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f377fe640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f380fb6c0 0x7f8f3810b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f36ffd640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f380fc990 0x7f8f3810be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 -- 172.21.15.172:0/3271141787 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f380fc990 msgr2=0x7f8f3810be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f380fc990 0x7f8f3810be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 -- 172.21.15.172:0/3271141787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f380fb6c0 msgr2=0x7f8f3810b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f380fb6c0 0x7f8f3810b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 -- 172.21.15.172:0/3271141787 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300098b0 con 0x7f8f380fd340 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f37fff640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 0x7f8f38112e90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009ca0 tx=0x7f8f30004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.272+0000 7f8f1ffff640 1 -- 172.21.15.172:0/3271141787 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30038070 con 0x7f8f380fd340 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.273+0000 7f8f1ffff640 1 -- 172.21.15.172:0/3271141787 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8f30033770 con 0x7f8f380fd340 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.273+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f38065ea0 con 0x7f8f380fd340 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.273+0000 7f8f1ffff640 1 -- 172.21.15.172:0/3271141787 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3003cb70 con 0x7f8f380fd340 2024-09-11T08:29:30.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.273+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f38066440 con 0x7f8f380fd340 2024-09-11T08:29:30.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.274+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f38106d30 con 0x7f8f380fd340 2024-09-11T08:29:30.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.274+0000 7f8f1ffff640 1 -- 172.21.15.172:0/3271141787 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7f8f30030070 con 0x7f8f380fd340 2024-09-11T08:29:30.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.275+0000 7f8f1ffff640 1 --2- 172.21.15.172:0/3271141787 >> v2:172.21.15.43:6800/1424089977 conn(0x7f8f2006eec0 0x7f8f20071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:30.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.275+0000 7f8f1ffff640 1 -- 172.21.15.172:0/3271141787 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 3785+0+0 (secure 0 0 0) 0x7f8f300adf80 con 0x7f8f380fd340 2024-09-11T08:29:30.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.275+0000 7f8f377fe640 1 --2- 172.21.15.172:0/3271141787 >> v2:172.21.15.43:6800/1424089977 conn(0x7f8f2006eec0 0x7f8f20071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:30.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.276+0000 7f8f377fe640 1 --2- 172.21.15.172:0/3271141787 >> v2:172.21.15.43:6800/1424089977 conn(0x7f8f2006eec0 0x7f8f20071370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f28004eb0 tx=0x7f8f2800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:30.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.279+0000 7f8f1ffff640 1 -- 172.21.15.172:0/3271141787 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8f300e02a0 con 0x7f8f380fd340 2024-09-11T08:29:30.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:30.443+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f38066660 con 0x7f8f2006eec0 2024-09-11T08:29:30.626 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:30 smithi172 ceph-mon[32987]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:30.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:30 smithi043 ceph-mon[29355]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:30 smithi043 ceph-mon[40592]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:31.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:31 smithi043 ceph-mon[29355]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:31.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:31 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:29:31.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:31 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:29:31.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:31 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:31.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:31 smithi043 ceph-mon[40592]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:31.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:31 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:29:31.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:31 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:29:31.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:31 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:31.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:31 smithi172 ceph-mon[32987]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:31.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:31 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:29:31.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:31 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:29:31.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:31 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:32.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:32 smithi172 ceph-mon[32987]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:32.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:32 smithi043 ceph-mon[29355]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:32.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:32 smithi043 ceph-mon[40592]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/3145869847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]: dispatch 2024-09-11T08:29:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]: dispatch 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]': finished 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[40592]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/76196702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/3145869847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]: dispatch 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]: dispatch 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]': finished 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[29355]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/76196702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:29:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/3145869847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]: dispatch 2024-09-11T08:29:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]: dispatch 2024-09-11T08:29:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23916580-474a-495f-9470-08f76d36069f"}]': finished 2024-09-11T08:29:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:33 smithi172 ceph-mon[32987]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:29:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:33 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/76196702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:29:34.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:34 smithi043 ceph-mon[40592]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:34 smithi043 ceph-mon[29355]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:34.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:34 smithi172 ceph-mon[32987]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:36.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:36 smithi172 ceph-mon[32987]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:36 smithi043 ceph-mon[40592]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:36.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:36 smithi043 ceph-mon[29355]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:38.560 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:38 smithi172 ceph-mon[32987]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:38 smithi043 ceph-mon[40592]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:38.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:38 smithi043 ceph-mon[29355]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:39.622 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:39 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:29:39.622 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:39 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:39 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:29:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:39 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:29:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:39 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:40.658 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:40 smithi172 ceph-mon[32987]: Deploying daemon osd.7 on smithi172 2024-09-11T08:29:40.658 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:40 smithi172 ceph-mon[32987]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:40.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:40 smithi043 ceph-mon[29355]: Deploying daemon osd.7 on smithi172 2024-09-11T08:29:40.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:40 smithi043 ceph-mon[29355]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:40.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:40 smithi043 ceph-mon[40592]: Deploying daemon osd.7 on smithi172 2024-09-11T08:29:40.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:40 smithi043 ceph-mon[40592]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:42.484 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:42 smithi172 ceph-mon[32987]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:42 smithi043 ceph-mon[40592]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:42.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:42 smithi043 ceph-mon[29355]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:44.671 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:44 smithi172 ceph-mon[32987]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:44.672 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:44 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:44.672 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:44 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:44.672 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:44 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:44 smithi043 ceph-mon[40592]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:44 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:44 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:44 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:44 smithi043 ceph-mon[29355]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:44 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:44 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:44 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:46.425 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:46 smithi172 ceph-mon[32987]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:46 smithi043 ceph-mon[29355]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:46 smithi043 ceph-mon[40592]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:47.406 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 7 on host 'smithi172' 2024-09-11T08:29:47.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.405+0000 7f8f1ffff640 1 -- 172.21.15.172:0/3271141787 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8f38066660 con 0x7f8f2006eec0 2024-09-11T08:29:47.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 >> v2:172.21.15.43:6800/1424089977 conn(0x7f8f2006eec0 msgr2=0x7f8f20071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:47.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3271141787 >> v2:172.21.15.43:6800/1424089977 conn(0x7f8f2006eec0 0x7f8f20071370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f28004eb0 tx=0x7f8f2800b040 comp rx=0 tx=0).stop 2024-09-11T08:29:47.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 msgr2=0x7f8f38112e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:47.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 0x7f8f38112e90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009ca0 tx=0x7f8f30004d10 comp rx=0 tx=0).stop 2024-09-11T08:29:47.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 shutdown_connections 2024-09-11T08:29:47.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3271141787 >> v2:172.21.15.43:6800/1424089977 conn(0x7f8f2006eec0 0x7f8f20071370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:47.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f380fc990 0x7f8f3810be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:47.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f380fd340 0x7f8f38112e90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:47.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 --2- 172.21.15.172:0/3271141787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f380fb6c0 0x7f8f3810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:47.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 >> 172.21.15.172:0/3271141787 conn(0x7f8f380f6ea0 msgr2=0x7f8f380f7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:47.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 shutdown_connections 2024-09-11T08:29:47.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:47.408+0000 7f8f3dbea640 1 -- 172.21.15.172:0/3271141787 wait complete. 2024-09-11T08:29:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='osd.7 v2:172.21.15.172:6812/1466581331' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:29:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:29:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:47.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:47.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:47.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:47.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:47.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:47 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.035 DEBUG:teuthology.orchestra.run.smithi172:osd.7> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T08:29:48.037 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T08:29:48.038 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T08:29:48.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='osd.7 v2:172.21.15.172:6812/1466581331' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:29:48.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:29:48.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:48.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:48.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='osd.7 v2:172.21.15.172:6812/1466581331' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:48.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:48.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:47 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:48.305 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:29:48.693 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[40592]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:48.693 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[40592]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:29:48.693 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[40592]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:29:48.693 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:48.693 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[40592]: from='osd.7 v2:172.21.15.172:6812/1466581331' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:48.693 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[40592]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:48.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[29355]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:48.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[29355]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:29:48.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[29355]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:29:48.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:48.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[29355]: from='osd.7 v2:172.21.15.172:6812/1466581331' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:48.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:48 smithi043 ceph-mon[29355]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:48 smithi172 ceph-mon[32987]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:48 smithi172 ceph-mon[32987]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:29:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:48 smithi172 ceph-mon[32987]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:29:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:48 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:48 smithi172 ceph-mon[32987]: from='osd.7 v2:172.21.15.172:6812/1466581331' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:48 smithi172 ceph-mon[32987]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T08:29:48.982 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:29:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:29:48.675+0000 7f6b33251640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T08:29:48.982 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:29:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:29:48.681+0000 7f6b3008c640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:29:49.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[29355]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[29355]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[29355]: osd.7 v2:172.21.15.172:6812/1466581331 boot 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[29355]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[40592]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[40592]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[40592]: osd.7 v2:172.21.15.172:6812/1466581331 boot 2024-09-11T08:29:49.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[40592]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:29:49.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:49 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:49 smithi172 ceph-mon[32987]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T08:29:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:49 smithi172 ceph-mon[32987]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T08:29:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:49 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:49 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:49 smithi172 ceph-mon[32987]: osd.7 v2:172.21.15.172:6812/1466581331 boot 2024-09-11T08:29:49.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:49 smithi172 ceph-mon[32987]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:29:49.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:49 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.151+0000 7f90b51b1640 1 -- 172.21.15.43:0/3274188392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fd170 msgr2=0x7f90b0103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.151+0000 7f90b51b1640 1 --2- 172.21.15.43:0/3274188392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fd170 0x7f90b0103d20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009cd0 tx=0x7f90a8004d10 comp rx=0 tx=0).stop 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.151+0000 7f90b51b1640 1 -- 172.21.15.43:0/3274188392 shutdown_connections 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.151+0000 7f90b51b1640 1 --2- 172.21.15.43:0/3274188392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fd170 0x7f90b0103d20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.151+0000 7f90b51b1640 1 --2- 172.21.15.43:0/3274188392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b00fc640 0x7f90b00fca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.151+0000 7f90b51b1640 1 --2- 172.21.15.43:0/3274188392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f90b00fbcf0 0x7f90b00fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.151+0000 7f90b51b1640 1 -- 172.21.15.43:0/3274188392 >> 172.21.15.43:0/3274188392 conn(0x7f90b00f6640 msgr2=0x7f90b00f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.152+0000 7f90b51b1640 1 -- 172.21.15.43:0/3274188392 shutdown_connections 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.152+0000 7f90b51b1640 1 -- 172.21.15.43:0/3274188392 wait complete. 2024-09-11T08:29:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.152+0000 7f90b51b1640 1 Processor -- start 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.152+0000 7f90b51b1640 1 -- start start 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.152+0000 7f90b51b1640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f90b00fbcf0 0x7f90b00fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.152+0000 7f90b51b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fc640 0x7f90b0111860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.152+0000 7f90b51b1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b00fd170 0x7f90b0113db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90b51b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0114540 con 0x7f90b00fc640 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90b51b1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01146b0 con 0x7f90b00fd170 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90b51b1640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0114870 con 0x7f90b00fbcf0 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f90b00fbcf0 0x7f90b00fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fc640 0x7f90b0111860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fc640 0x7f90b0111860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41740/0 (socket says 172.21.15.43:41740) 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f90b00fbcf0 0x7f90b00fb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49668/0 (socket says 172.21.15.43:49668) 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90ae59c640 1 -- 172.21.15.43:0/1808130689 learned_addr learned my addr 172.21.15.43:0/1808130689 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:29:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90af59e640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b00fd170 0x7f90b0113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90aed9d640 1 -- 172.21.15.43:0/1808130689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b00fd170 msgr2=0x7f90b0113db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90aed9d640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b00fd170 0x7f90b0113db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90aed9d640 1 -- 172.21.15.43:0/1808130689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fc640 msgr2=0x7f90b0111860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90aed9d640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fc640 0x7f90b0111860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90aed9d640 1 -- 172.21.15.43:0/1808130689 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0005040 con 0x7f90b00fbcf0 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90ae59c640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fc640 0x7f90b0111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90aed9d640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f90b00fbcf0 0x7f90b00fb3a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009400 tx=0x7f90a0009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90977fe640 1 -- 172.21.15.43:0/1808130689 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0007bb0 con 0x7f90b00fbcf0 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90977fe640 1 -- 172.21.15.43:0/1808130689 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f90a000f040 con 0x7f90b00fbcf0 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a80098b0 con 0x7f90b00fbcf0 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.153+0000 7f90977fe640 1 -- 172.21.15.43:0/1808130689 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0013ba0 con 0x7f90b00fbcf0 2024-09-11T08:29:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.154+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b0114e80 con 0x7f90b00fbcf0 2024-09-11T08:29:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.155+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b004bf60 con 0x7f90b00fbcf0 2024-09-11T08:29:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.155+0000 7f90977fe640 1 -- 172.21.15.43:0/1808130689 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7f90a000e8d0 con 0x7f90b00fbcf0 2024-09-11T08:29:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.156+0000 7f90977fe640 1 --2- 172.21.15.43:0/1808130689 >> v2:172.21.15.43:6800/1424089977 conn(0x7f909806ee90 0x7f9098071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.156+0000 7f90977fe640 1 -- 172.21.15.43:0/1808130689 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (secure 0 0 0) 0x7f90a008d940 con 0x7f90b00fbcf0 2024-09-11T08:29:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.156+0000 7f90ae59c640 1 --2- 172.21.15.43:0/1808130689 >> v2:172.21.15.43:6800/1424089977 conn(0x7f909806ee90 0x7f9098071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.156+0000 7f90ae59c640 1 --2- 172.21.15.43:0/1808130689 >> v2:172.21.15.43:6800/1424089977 conn(0x7f909806ee90 0x7f9098071340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90b00fb150 tx=0x7f90a400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.160+0000 7f90977fe640 1 -- 172.21.15.43:0/1808130689 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f90a005b140 con 0x7f90b00fbcf0 2024-09-11T08:29:50.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.327+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f90b0106610 con 0x7f90b00fbcf0 2024-09-11T08:29:50.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.328+0000 7f90977fe640 1 -- 172.21.15.43:0/1808130689 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v50) v1 ==== 74+0+130 (secure 0 0 0) 0x7f90a005aae0 con 0x7f90b00fbcf0 2024-09-11T08:29:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:29:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.329+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 >> v2:172.21.15.43:6800/1424089977 conn(0x7f909806ee90 msgr2=0x7f9098071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.329+0000 7f90b51b1640 1 --2- 172.21.15.43:0/1808130689 >> v2:172.21.15.43:6800/1424089977 conn(0x7f909806ee90 0x7f9098071340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90b00fb150 tx=0x7f90a400b040 comp rx=0 tx=0).stop 2024-09-11T08:29:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.329+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f90b00fbcf0 msgr2=0x7f90b00fb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f90b00fbcf0 0x7f90b00fb3a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009400 tx=0x7f90a0009da0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 shutdown_connections 2024-09-11T08:29:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 --2- 172.21.15.43:0/1808130689 >> v2:172.21.15.43:6800/1424089977 conn(0x7f909806ee90 0x7f9098071340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b00fd170 0x7f90b0113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b00fc640 0x7f90b0111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 --2- 172.21.15.43:0/1808130689 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f90b00fbcf0 0x7f90b00fb3a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 >> 172.21.15.43:0/1808130689 conn(0x7f90b00f6640 msgr2=0x7f90b00f74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 shutdown_connections 2024-09-11T08:29:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:50.330+0000 7f90b51b1640 1 -- 172.21.15.43:0/1808130689 wait complete. 2024-09-11T08:29:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[40592]: purged_snaps scrub starts 2024-09-11T08:29:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[40592]: purged_snaps scrub ok 2024-09-11T08:29:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[40592]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[40592]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:29:50.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[40592]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:29:50.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1808130689' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:29:50.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[29355]: purged_snaps scrub starts 2024-09-11T08:29:50.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[29355]: purged_snaps scrub ok 2024-09-11T08:29:50.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[29355]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:50.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[29355]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:29:50.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[29355]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:29:50.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1808130689' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:29:50.952 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1726043388,"num_in_osds":8,"osd_in_since":1726043372,"num_remapped_pgs":0} 2024-09-11T08:29:50.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:29:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:50 smithi172 ceph-mon[32987]: purged_snaps scrub starts 2024-09-11T08:29:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:50 smithi172 ceph-mon[32987]: purged_snaps scrub ok 2024-09-11T08:29:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:50 smithi172 ceph-mon[32987]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:29:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:50 smithi172 ceph-mon[32987]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:29:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:50 smithi172 ceph-mon[32987]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:29:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1808130689' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:29:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:29:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: Detected new or changed devices on smithi172 2024-09-11T08:29:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:29:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[29355]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: Detected new or changed devices on smithi172 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:51.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:51.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:29:51.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:51 smithi043 ceph-mon[40592]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:52.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: Detected new or changed devices on smithi172 2024-09-11T08:29:52.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:52.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:52.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:29:52.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:51 smithi172 ceph-mon[32987]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 -- 172.21.15.43:0/798902961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fb6f0 msgr2=0x7fde800fbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 --2- 172.21.15.43:0/798902961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fb6f0 0x7fde800fbb60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde70004980 tx=0x7fde70009da0 comp rx=0 tx=0).stop 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 -- 172.21.15.43:0/798902961 shutdown_connections 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 --2- 172.21.15.43:0/798902961 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde800fc0a0 0x7fde801084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 --2- 172.21.15.43:0/798902961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fb6f0 0x7fde800fbb60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 --2- 172.21.15.43:0/798902961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde80101710 0x7fde80101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 -- 172.21.15.43:0/798902961 >> 172.21.15.43:0/798902961 conn(0x7fde80071390 msgr2=0x7fde800f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 -- 172.21.15.43:0/798902961 shutdown_connections 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 -- 172.21.15.43:0/798902961 wait complete. 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 Processor -- start 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.858+0000 7fde86d23640 1 -- start start 2024-09-11T08:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde86d23640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde800fb6f0 0x7fde800fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde86d23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fc0a0 0x7fde800fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde86d23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde80101710 0x7fde8011b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde86d23640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800fe0a0 con 0x7fde800fc0a0 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde86d23640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8011b7e0 con 0x7fde80101710 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde86d23640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8011b920 con 0x7fde800fb6f0 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde800fb6f0 0x7fde800fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fc0a0 0x7fde800fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde800fb6f0 0x7fde800fd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:47562/0 (socket says 172.21.15.43:47562) 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 -- 172.21.15.43:0/3206978526 learned_addr learned my addr 172.21.15.43:0/3206978526 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde852c0640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde80101710 0x7fde8011b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 -- 172.21.15.43:0/3206978526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde80101710 msgr2=0x7fde8011b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde80101710 0x7fde8011b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 -- 172.21.15.43:0/3206978526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fc0a0 msgr2=0x7fde800fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fc0a0 0x7fde800fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 -- 172.21.15.43:0/3206978526 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70005040 con 0x7fde800fb6f0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde852c0640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde80101710 0x7fde8011b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde77fff640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fc0a0 0x7fde800fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.859+0000 7fde84abf640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde800fb6f0 0x7fde800fd300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00c890 tx=0x7fde7c0043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.860+0000 7fde757fa640 1 -- 172.21.15.43:0/3206978526 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00ce20 con 0x7fde800fb6f0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.860+0000 7fde757fa640 1 -- 172.21.15.43:0/3206978526 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fde7c00acc0 con 0x7fde800fb6f0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.860+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8011bb80 con 0x7fde800fb6f0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.860+0000 7fde757fa640 1 -- 172.21.15.43:0/3206978526 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c012ca0 con 0x7fde800fb6f0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.860+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8011c050 con 0x7fde800fb6f0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.861+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde800fce30 con 0x7fde800fb6f0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.861+0000 7fde757fa640 1 -- 172.21.15.43:0/3206978526 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7fde7c00a850 con 0x7fde800fb6f0 2024-09-11T08:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.862+0000 7fde757fa640 1 --2- 172.21.15.43:0/3206978526 >> v2:172.21.15.43:6800/1424089977 conn(0x7fde6806ede0 0x7fde68071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:52.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.862+0000 7fde77fff640 1 --2- 172.21.15.43:0/3206978526 >> v2:172.21.15.43:6800/1424089977 conn(0x7fde6806ede0 0x7fde68071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:52.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.862+0000 7fde757fa640 1 -- 172.21.15.43:0/3206978526 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4077+0+0 (secure 0 0 0) 0x7fde7c08dfc0 con 0x7fde800fb6f0 2024-09-11T08:29:52.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.862+0000 7fde77fff640 1 --2- 172.21.15.43:0/3206978526 >> v2:172.21.15.43:6800/1424089977 conn(0x7fde6806ede0 0x7fde68071290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde800fec40 tx=0x7fde70009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:52.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:52.866+0000 7fde757fa640 1 -- 172.21.15.43:0/3206978526 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fde7c05b650 con 0x7fde800fb6f0 2024-09-11T08:29:53.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.030+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde8010aaf0 con 0x7fde800fb6f0 2024-09-11T08:29:53.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.031+0000 7fde757fa640 1 -- 172.21.15.43:0/3206978526 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+11149 (secure 0 0 0) 0x7fde7c016090 con 0x7fde800fb6f0 2024-09-11T08:29:53.031 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:29:53.031 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":51,"fsid":"327ee848-7017-11ef-bcea-c7b262605968","created":"2024-09-11T08:24:43.166587+0000","modified":"2024-09-11T08:29:50.823104+0000","last_up_change":"2024-09-11T08:29:48.817615+0000","last_in_change":"2024-09-11T08:29:32.826492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:27:53.800096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0752b9ca-c21a-45f5-9974-5942fd4e17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":3394646540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3394646540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3394646540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6803","nonce":3394646540}]},"public_addr":"172.21.15.43:6801/3394646540","cluster_addr":"172.21.15.43:6802/3394646540","heartbeat_back_addr":"172.21.15.43:6804/3394646540","heartbeat_front_addr":"172.21.15.43:6803/3394646540","state":["exists","up"]},{"osd":1,"uuid":"039df926-3c4b-48cd-9141-3a7684037575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":2076458881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2076458881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2076458881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6807","nonce":2076458881}]},"public_addr":"172.21.15.43:6805/2076458881","cluster_addr":"172.21.15.43:6806/2076458881","heartbeat_back_addr":"172.21.15.43:6808/2076458881","heartbeat_front_addr":"172.21.15.43:6807/2076458881","state":["exists","up"]},{"osd":2,"uuid":"d56b0955-f9bc-40f9-9fca-78ef5af1fd77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6809","nonce":319507648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":319507648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":319507648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":319507648}]},"public_addr":"172.21.15.43:6809/319507648","cluster_addr":"172.21.15.43:6810/319507648","heartbeat_back_addr":"172.21.15.43:6812/319507648","heartbeat_front_addr":"172.21.15.43:6811/319507648","state":["exists","up"]},{"osd":3,"uuid":"f653df6d-614c-4e93-b9dc-3f164fba74d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":87557626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":87557626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":87557626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":87557626}]},"public_addr":"172.21.15.43:6813/87557626","cluster_addr":"172.21.15.43:6814/87557626","heartbeat_back_addr":"172.21.15.43:6816/87557626","heartbeat_front_addr":"172.21.15.43:6815/87557626","state":["exists","up"]},{"osd":4,"uuid":"3083c1a6-6c94-47ed-bb13-86690fcebd11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":2140228578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6801","nonce":2140228578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6803","nonce":2140228578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2140228578}]},"public_addr":"172.21.15.172:6800/2140228578","cluster_addr":"172.21.15.172:6801/2140228578","heartbeat_back_addr":"172.21.15.172:6803/2140228578","heartbeat_front_addr":"172.21.15.172:6802/2140228578","state":["exists","up"]},{"osd":5,"uuid":"dab026a9-396f-4828-8d32-9c7e6142d1b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2848105378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6805","nonce":2848105378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6807","nonce":2848105378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2848105378}]},"public_addr":"172.21.15.172:6804/2848105378","cluster_addr":"172.21.15.172:6805/2848105378","heartbeat_back_addr":"172.21.15.172:6807/2848105378","heartbeat_front_addr":"172.21.15.172:6806/2848105378","state":["exists","up"]},{"osd":6,"uuid":"06710545-d1ff-4929-8dbe-97868b7dea5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2701192652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6809","nonce":2701192652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6811","nonce":2701192652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2701192652}]},"public_addr":"172.21.15.172:6808/2701192652","cluster_addr":"172.21.15.172:6809/2701192652","heartbeat_back_addr":"172.21.15.172:6811/2701192652","heartbeat_front_addr":"172.21.15.172:6810/2701192652","state":["exists","up"]},{"osd":7,"uuid":"23916580-474a-495f-9470-08f76d36069f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1466581331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6813","nonce":1466581331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6815","nonce":1466581331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1466581331}]},"public_addr":"172.21.15.172:6812/1466581331","cluster_addr":"172.21.15.172:6813/1466581331","heartbeat_back_addr":"172.21.15.172:6815/1466581331","heartbeat_front_addr":"172.21.15.172:6814/1466581331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:01.839914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:25.733653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:49.695899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:14.885633+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:36.661956+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:59.899466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:29:23.577485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:29:47.969389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6800/225101565":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/1607164598":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/162481557":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/848421812":"2024-09-12T08:25:15.636368+0000","172.21.15.43:0/1657974672":"2024-09-12T08:25:15.636368+0000","172.21.15.43:0/2577594813":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/2596169613":"2024-09-12T08:25:15.636368+0000","172.21.15.43:6800/3257998337":"2024-09-12T08:25:15.636368+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:29:53.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 >> v2:172.21.15.43:6800/1424089977 conn(0x7fde6806ede0 msgr2=0x7fde68071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:53.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 --2- 172.21.15.43:0/3206978526 >> v2:172.21.15.43:6800/1424089977 conn(0x7fde6806ede0 0x7fde68071290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde800fec40 tx=0x7fde70009da0 comp rx=0 tx=0).stop 2024-09-11T08:29:53.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde800fb6f0 msgr2=0x7fde800fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde800fb6f0 0x7fde800fd300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00c890 tx=0x7fde7c0043c0 comp rx=0 tx=0).stop 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 shutdown_connections 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 --2- 172.21.15.43:0/3206978526 >> v2:172.21.15.43:6800/1424089977 conn(0x7fde6806ede0 0x7fde68071290 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde80101710 0x7fde8011b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde800fc0a0 0x7fde800fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 --2- 172.21.15.43:0/3206978526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde800fb6f0 0x7fde800fd300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 >> 172.21.15.43:0/3206978526 conn(0x7fde80071390 msgr2=0x7fde80065110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 shutdown_connections 2024-09-11T08:29:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:53.033+0000 7fde86d23640 1 -- 172.21.15.43:0/3206978526 wait complete. 2024-09-11T08:29:53.326 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3206978526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:29:53.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3206978526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:29:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3206978526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:29:53.731 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:27:53.800096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T08:29:53.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T08:29:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:29:54.372 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:54 smithi043 ceph-mon[40592]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:54.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:54 smithi043 ceph-mon[29355]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:54.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:54 smithi172 ceph-mon[32987]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3540320483 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb80fad70 msgr2=0x7f7cb80fb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3540320483 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb80fad70 0x7f7cb80fb1c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7cac004980 tx=0x7f7cac009da0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3540320483 shutdown_connections 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3540320483 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb80fad70 0x7f7cb80fb1c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3540320483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb80fb700 0x7f7cb8109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3540320483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb8102880 0x7f7cb8102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3540320483 >> 172.21.15.43:0/3540320483 conn(0x7f7cb8074c80 msgr2=0x7f7cb80f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3540320483 shutdown_connections 2024-09-11T08:29:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.572+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3540320483 wait complete. 2024-09-11T08:29:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.573+0000 7f7cbeff3640 1 Processor -- start 2024-09-11T08:29:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.573+0000 7f7cbeff3640 1 -- start start 2024-09-11T08:29:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.573+0000 7f7cbeff3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb80fad70 0x7f7cb8112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.573+0000 7f7cbeff3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb80fb700 0x7f7cb81132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.573+0000 7f7cbeff3640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb8102880 0x7f7cb810cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.573+0000 7f7cbeff3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb810d560 con 0x7f7cb80fad70 2024-09-11T08:29:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.573+0000 7f7cbeff3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb810d6d0 con 0x7f7cb80fb700 2024-09-11T08:29:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.573+0000 7f7cbeff3640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb810d890 con 0x7f7cb8102880 2024-09-11T08:29:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb80fad70 0x7f7cb8112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbd590640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb8102880 0x7f7cb810cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb80fad70 0x7f7cb8112d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53312/0 (socket says 172.21.15.43:53312) 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 -- 172.21.15.43:0/3692984712 learned_addr learned my addr 172.21.15.43:0/3692984712 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cb7fff640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb80fb700 0x7f7cb81132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 -- 172.21.15.43:0/3692984712 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb8102880 msgr2=0x7f7cb810cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb8102880 0x7f7cb810cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 -- 172.21.15.43:0/3692984712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb80fb700 msgr2=0x7f7cb81132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb80fb700 0x7f7cb81132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 -- 172.21.15.43:0/3692984712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00098b0 con 0x7f7cb80fad70 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbd590640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb8102880 0x7f7cb810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbcd8f640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb80fad70 0x7f7cb8112d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800ebf0 tx=0x7f7ca800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cb57fa640 1 -- 172.21.15.43:0/3692984712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8007c30 con 0x7f7cb80fad70 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cb57fa640 1 -- 172.21.15.43:0/3692984712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f7ca800f040 con 0x7f7cb80fad70 2024-09-11T08:29:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cb57fa640 1 -- 172.21.15.43:0/3692984712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8009340 con 0x7f7cb80fad70 2024-09-11T08:29:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.574+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac005040 con 0x7f7cb80fad70 2024-09-11T08:29:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.575+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb8119bd0 con 0x7f7cb80fad70 2024-09-11T08:29:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.575+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb80fe090 con 0x7f7cb80fad70 2024-09-11T08:29:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.576+0000 7f7cb57fa640 1 -- 172.21.15.43:0/3692984712 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7f7ca8009730 con 0x7f7cb80fad70 2024-09-11T08:29:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.577+0000 7f7cb57fa640 1 --2- 172.21.15.43:0/3692984712 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7ca006ee90 0x7f7ca0071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.577+0000 7f7cb57fa640 1 -- 172.21.15.43:0/3692984712 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4077+0+0 (secure 0 0 0) 0x7f7ca808d990 con 0x7f7cb80fad70 2024-09-11T08:29:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.577+0000 7f7cb7fff640 1 --2- 172.21.15.43:0/3692984712 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7ca006ee90 0x7f7ca0071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.577+0000 7f7cb7fff640 1 --2- 172.21.15.43:0/3692984712 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7ca006ee90 0x7f7ca0071340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009f80 tx=0x7f7cac009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:55.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.581+0000 7f7cb57fa640 1 -- 172.21.15.43:0/3692984712 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f7ca805b0d0 con 0x7f7cb80fad70 2024-09-11T08:29:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:55 smithi043 ceph-mon[29355]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:29:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:55 smithi043 ceph-mon[29355]: Cluster is now healthy 2024-09-11T08:29:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:55 smithi043 ceph-mon[40592]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:29:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:55 smithi043 ceph-mon[40592]: Cluster is now healthy 2024-09-11T08:29:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:55 smithi172 ceph-mon[32987]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:29:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:55 smithi172 ceph-mon[32987]: Cluster is now healthy 2024-09-11T08:29:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.748+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7cb810bd80 con 0x7f7cb80fad70 2024-09-11T08:29:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.749+0000 7f7cb57fa640 1 -- 172.21.15.43:0/3692984712 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ca805aa70 con 0x7f7cb80fad70 2024-09-11T08:29:55.749 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7ca006ee90 msgr2=0x7f7ca0071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3692984712 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7ca006ee90 0x7f7ca0071340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009f80 tx=0x7f7cac009da0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb80fad70 msgr2=0x7f7cb8112d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb80fad70 0x7f7cb8112d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800ebf0 tx=0x7f7ca800b040 comp rx=0 tx=0).stop 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 shutdown_connections 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3692984712 >> v2:172.21.15.43:6800/1424089977 conn(0x7f7ca006ee90 0x7f7ca0071340 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7cb8102880 0x7f7cb810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb80fb700 0x7f7cb81132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 --2- 172.21.15.43:0/3692984712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb80fad70 0x7f7cb8112d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.750+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 >> 172.21.15.43:0/3692984712 conn(0x7f7cb8074c80 msgr2=0x7f7cb8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.751+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 shutdown_connections 2024-09-11T08:29:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:29:55.751+0000 7f7cbeff3640 1 -- 172.21.15.43:0/3692984712 wait complete. 2024-09-11T08:29:56.464 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi043 2024-09-11T08:29:56.464 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi043=foo.a' 2024-09-11T08:29:56.720 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:29:56.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:56 smithi172 ceph-mon[32987]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:56.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3692984712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:29:56.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:56 smithi043 ceph-mon[29355]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:56.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3692984712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:29:56.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:56 smithi043 ceph-mon[40592]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:29:56.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3692984712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:29:58.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 -- 172.21.15.172:0/668233728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fcd90 msgr2=0x7fb8680fd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:58.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/668233728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fcd90 0x7fb8680fd1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb858004950 tx=0x7fb858009da0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 -- 172.21.15.172:0/668233728 shutdown_connections 2024-09-11T08:29:58.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/668233728 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb8680fbd50 0x7fb868103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/668233728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8680fb3a0 0x7fb8680fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/668233728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fcd90 0x7fb8680fd1a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 -- 172.21.15.172:0/668233728 >> 172.21.15.172:0/668233728 conn(0x7fb8680f6e70 msgr2=0x7fb8680f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 -- 172.21.15.172:0/668233728 shutdown_connections 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.326+0000 7fb86cdfd640 1 -- 172.21.15.172:0/668233728 wait complete. 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86cdfd640 1 Processor -- start 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86cdfd640 1 -- start start 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86cdfd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8680fb3a0 0x7fb86810d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86cdfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fbd50 0x7fb86810d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86cdfd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb8680fcd90 0x7fb86810e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86cdfd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86810e9e0 con 0x7fb8680fb3a0 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86cdfd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86810eb50 con 0x7fb8680fbd50 2024-09-11T08:29:58.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86cdfd640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86810ed10 con 0x7fb8680fcd90 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb865d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fbd50 0x7fb86810d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb865d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fbd50 0x7fb86810d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44320/0 (socket says 172.21.15.172:44320) 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb865d9b640 1 -- 172.21.15.172:0/1618029267 learned_addr learned my addr 172.21.15.172:0/1618029267 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb865d9b640 1 -- 172.21.15.172:0/1618029267 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb8680fcd90 msgr2=0x7fb86810e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb866d9d640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb8680fcd90 0x7fb86810e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb86659c640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8680fb3a0 0x7fb86810d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb865d9b640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb8680fcd90 0x7fb86810e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb865d9b640 1 -- 172.21.15.172:0/1618029267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8680fb3a0 msgr2=0x7fb86810d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb865d9b640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8680fb3a0 0x7fb86810d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.327+0000 7fb865d9b640 1 -- 172.21.15.172:0/1618029267 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858005040 con 0x7fb8680fbd50 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.328+0000 7fb865d9b640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fbd50 0x7fb86810d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b810 tx=0x7fb8500091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.328+0000 7fb85effd640 1 -- 172.21.15.172:0/1618029267 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85000be60 con 0x7fb8680fbd50 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.328+0000 7fb85effd640 1 -- 172.21.15.172:0/1618029267 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fb85000cba0 con 0x7fb8680fbd50 2024-09-11T08:29:58.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.328+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86810eff0 con 0x7fb8680fbd50 2024-09-11T08:29:58.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.328+0000 7fb85effd640 1 -- 172.21.15.172:0/1618029267 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850015b60 con 0x7fb8680fbd50 2024-09-11T08:29:58.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.328+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb868115780 con 0x7fb8680fbd50 2024-09-11T08:29:58.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.329+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb868106da0 con 0x7fb8680fbd50 2024-09-11T08:29:58.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.329+0000 7fb85effd640 1 -- 172.21.15.172:0/1618029267 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7fb850012070 con 0x7fb8680fbd50 2024-09-11T08:29:58.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.330+0000 7fb85effd640 1 --2- 172.21.15.172:0/1618029267 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ede0 0x7fb854071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:29:58.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.330+0000 7fb85effd640 1 -- 172.21.15.172:0/1618029267 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4077+0+0 (secure 0 0 0) 0x7fb85008d300 con 0x7fb8680fbd50 2024-09-11T08:29:58.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.330+0000 7fb86659c640 1 --2- 172.21.15.172:0/1618029267 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ede0 0x7fb854071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:29:58.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.331+0000 7fb86659c640 1 --2- 172.21.15.172:0/1618029267 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ede0 0x7fb854071290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8580043d0 tx=0x7fb858009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:29:58.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.334+0000 7fb85effd640 1 -- 172.21.15.172:0/1618029267 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb85005aa10 con 0x7fb8680fbd50 2024-09-11T08:29:58.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.498+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb86804b790 con 0x7fb85406ede0 2024-09-11T08:29:58.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.505+0000 7fb85effd640 1 -- 172.21.15.172:0/1618029267 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7fb86804b790 con 0x7fb85406ede0 2024-09-11T08:29:58.505 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled rgw.foo.a update... 2024-09-11T08:29:58.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ede0 msgr2=0x7fb854071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:58.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/1618029267 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ede0 0x7fb854071290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8580043d0 tx=0x7fb858009da0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fbd50 msgr2=0x7fb86810d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fbd50 0x7fb86810d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b810 tx=0x7fb8500091d0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 shutdown_connections 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/1618029267 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ede0 0x7fb854071290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb8680fcd90 0x7fb86810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8680fbd50 0x7fb86810d570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 --2- 172.21.15.172:0/1618029267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8680fb3a0 0x7fb86810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 >> 172.21.15.172:0/1618029267 conn(0x7fb8680f6e70 msgr2=0x7fb8680f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 shutdown_connections 2024-09-11T08:29:58.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:29:58.507+0000 7fb86cdfd640 1 -- 172.21.15.172:0/1618029267 wait complete. 2024-09-11T08:29:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:58 smithi172 ceph-mon[32987]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:29:58.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:58 smithi043 ceph-mon[40592]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:29:58.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:58 smithi043 ceph-mon[29355]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:29:59.134 DEBUG:teuthology.orchestra.run.smithi043:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T08:29:59.137 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi172 2024-09-11T08:29:59.137 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T08:29:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:29:59.753 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:59.753 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-11T08:29:59.753 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.753 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:59.753 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:59.753 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:59.753 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.754 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:29:59.754 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:29:59.754 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.754 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:29:59 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:59.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:59.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-11T08:29:59.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:29:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:29:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:29:59 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:00 smithi043 ceph-mon[40592]: Deploying daemon rgw.foo.a on smithi043 2024-09-11T08:30:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:00 smithi043 ceph-mon[40592]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T08:30:00.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:00 smithi043 ceph-mon[40592]: overall HEALTH_OK 2024-09-11T08:30:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:00 smithi043 ceph-mon[29355]: Deploying daemon rgw.foo.a on smithi043 2024-09-11T08:30:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:00 smithi043 ceph-mon[29355]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T08:30:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:00 smithi043 ceph-mon[29355]: overall HEALTH_OK 2024-09-11T08:30:00.727 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:00 smithi172 ceph-mon[32987]: Deploying daemon rgw.foo.a on smithi043 2024-09-11T08:30:00.727 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:00 smithi172 ceph-mon[32987]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T08:30:00.727 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:00 smithi172 ceph-mon[32987]: overall HEALTH_OK 2024-09-11T08:30:00.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 -- 172.21.15.172:0/905895713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fe770 msgr2=0x7f123c0fa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 --2- 172.21.15.172:0/905895713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fe770 0x7f123c0fa8d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1230004950 tx=0x7f1230009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 -- 172.21.15.172:0/905895713 shutdown_connections 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 --2- 172.21.15.172:0/905895713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f123c0fb000 0x7f123c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 --2- 172.21.15.172:0/905895713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fe770 0x7f123c0fa8d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 --2- 172.21.15.172:0/905895713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c0fde20 0x7f123c0fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 -- 172.21.15.172:0/905895713 >> 172.21.15.172:0/905895713 conn(0x7f123c0f6640 msgr2=0x7f123c0f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 -- 172.21.15.172:0/905895713 shutdown_connections 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.941+0000 7f124290b640 1 -- 172.21.15.172:0/905895713 wait complete. 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f124290b640 1 Processor -- start 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f124290b640 1 -- start start 2024-09-11T08:30:00.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f124290b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c0fb000 0x7f123c0fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f124290b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fde20 0x7f123c0fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f124290b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f123c0fe770 0x7f123c0fc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f124290b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0fc780 con 0x7f123c0fb000 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f124290b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0fc8f0 con 0x7f123c0fde20 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f124290b640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c116d80 con 0x7f123c0fe770 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f123b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fde20 0x7f123c0fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f123b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fde20 0x7f123c0fbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44338/0 (socket says 172.21.15.172:44338) 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f123b7fe640 1 -- 172.21.15.172:0/2862601802 learned_addr learned my addr 172.21.15.172:0/2862601802 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f123bfff640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c0fb000 0x7f123c0fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.942+0000 7f1240ea8640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f123c0fe770 0x7f123c0fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f123b7fe640 1 -- 172.21.15.172:0/2862601802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f123c0fe770 msgr2=0x7f123c0fc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f123b7fe640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f123c0fe770 0x7f123c0fc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:00.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f123b7fe640 1 -- 172.21.15.172:0/2862601802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c0fb000 msgr2=0x7f123c0fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f123b7fe640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c0fb000 0x7f123c0fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f123b7fe640 1 -- 172.21.15.172:0/2862601802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1230005040 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f123b7fe640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fde20 0x7f123c0fbb00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1230009f80 tx=0x7f1230009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f1238ff9640 1 -- 172.21.15.172:0/2862601802 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230000c90 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f1238ff9640 1 -- 172.21.15.172:0/2862601802 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f1230032a50 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c116f80 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f1238ff9640 1 -- 172.21.15.172:0/2862601802 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230033670 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.943+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c117440 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.944+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c04bf60 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.945+0000 7f1238ff9640 1 -- 172.21.15.172:0/2862601802 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7f1230030020 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.945+0000 7f1238ff9640 1 --2- 172.21.15.172:0/2862601802 >> v2:172.21.15.43:6800/1424089977 conn(0x7f122406ee90 0x7f1224071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.945+0000 7f1238ff9640 1 -- 172.21.15.172:0/2862601802 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4077+0+0 (secure 0 0 0) 0x7f12300ad620 con 0x7f123c0fde20 2024-09-11T08:30:00.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.945+0000 7f123bfff640 1 --2- 172.21.15.172:0/2862601802 >> v2:172.21.15.43:6800/1424089977 conn(0x7f122406ee90 0x7f1224071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:00.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.946+0000 7f123bfff640 1 --2- 172.21.15.172:0/2862601802 >> v2:172.21.15.43:6800/1424089977 conn(0x7f122406ee90 0x7f1224071340 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f122c004eb0 tx=0x7f122c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:00.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:00.949+0000 7f1238ff9640 1 -- 172.21.15.172:0/2862601802 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f123007aea0 con 0x7f123c0fde20 2024-09-11T08:30:01.090 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:30:00 smithi043 podman[68308]: 2024-09-11 08:30:00.975186016 +0000 UTC m=+0.130584596 container create 9564a4bb9567b7f4d9dd3b44ba4f934640c8640471c746ff52f44227a7041a22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T08:30:01.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.104+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f123c103500 con 0x7f123c0fde20 2024-09-11T08:30:01.644 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:30:01 smithi043 podman[68308]: 2024-09-11 08:30:01.211123521 +0000 UTC m=+0.366522111 container init 9564a4bb9567b7f4d9dd3b44ba4f934640c8640471c746ff52f44227a7041a22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T08:30:01.644 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:30:01 smithi043 podman[68308]: 2024-09-11 08:30:01.216005183 +0000 UTC m=+0.371403763 container start 9564a4bb9567b7f4d9dd3b44ba4f934640c8640471c746ff52f44227a7041a22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:30:01.644 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:30:01 smithi043 bash[68308]: 9564a4bb9567b7f4d9dd3b44ba4f934640c8640471c746ff52f44227a7041a22 2024-09-11T08:30:01.645 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:30:01 smithi043 systemd[1]: Started Ceph rgw.foo.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:30:01.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.976+0000 7f1238ff9640 1 -- 172.21.15.172:0/2862601802 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v52) v1 ==== 160+0+0 (secure 0 0 0) 0x7f123007a840 con 0x7f123c0fde20 2024-09-11T08:30:01.978 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'datapool' created 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 >> v2:172.21.15.43:6800/1424089977 conn(0x7f122406ee90 msgr2=0x7f1224071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 --2- 172.21.15.172:0/2862601802 >> v2:172.21.15.43:6800/1424089977 conn(0x7f122406ee90 0x7f1224071340 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f122c004eb0 tx=0x7f122c00b040 comp rx=0 tx=0).stop 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fde20 msgr2=0x7f123c0fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fde20 0x7f123c0fbb00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1230009f80 tx=0x7f1230009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 shutdown_connections 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 --2- 172.21.15.172:0/2862601802 >> v2:172.21.15.43:6800/1424089977 conn(0x7f122406ee90 0x7f1224071340 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f123c0fe770 0x7f123c0fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f123c0fde20 0x7f123c0fbb00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 --2- 172.21.15.172:0/2862601802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f123c0fb000 0x7f123c0fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.980+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 >> 172.21.15.172:0/2862601802 conn(0x7f123c0f6640 msgr2=0x7f123c0f7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.981+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 shutdown_connections 2024-09-11T08:30:01.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:01.981+0000 7f124290b640 1 -- 172.21.15.172:0/2862601802 wait complete. 2024-09-11T08:30:02.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/2862601802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:30:02.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/2862601802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:30:02.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/2862601802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:30:02.606 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T08:30:02.851 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:30:03.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:02 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:03.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:30:03.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-11T08:30:03.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:03.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:02 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:04.328 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:30:04.329 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:04 smithi172 ceph-mon[32987]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:30:04.329 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:04.329 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:04.329 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:04.329 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:04 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:04.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:30:04.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[40592]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:30:04.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[29355]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:04 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: pgmap v175: 36 pgs: 13 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: Checking dashboard <-> RGW credentials 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3550179027' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/1331824501' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:30:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: pgmap v175: 36 pgs: 13 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: Checking dashboard <-> RGW credentials 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3550179027' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/1331824501' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:30:05.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:30:05.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: pgmap v175: 36 pgs: 13 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: Checking dashboard <-> RGW credentials 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3550179027' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/1331824501' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:30:05.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:30:06.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:30:06.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[29355]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:30:06.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:30:06.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[29355]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:30:06.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[29355]: pgmap v178: 68 pgs: 2 creating+activating, 13 creating+peering, 29 unknown, 24 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:30:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:30:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[40592]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:30:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:30:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[40592]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:30:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:06 smithi043 ceph-mon[40592]: pgmap v178: 68 pgs: 2 creating+activating, 13 creating+peering, 29 unknown, 24 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:30:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:30:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:06 smithi172 ceph-mon[32987]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:30:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:30:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:06 smithi172 ceph-mon[32987]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:30:06.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:06 smithi172 ceph-mon[32987]: pgmap v178: 68 pgs: 2 creating+activating, 13 creating+peering, 29 unknown, 24 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:30:07.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[29355]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:30:07.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[29355]: Cluster is now healthy 2024-09-11T08:30:07.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[29355]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3550179027' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[29355]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[40592]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[40592]: Cluster is now healthy 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[40592]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3550179027' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:07 smithi043 ceph-mon[40592]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.480 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:07 smithi172 ceph-mon[32987]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:30:07.480 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:07 smithi172 ceph-mon[32987]: Cluster is now healthy 2024-09-11T08:30:07.480 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:07 smithi172 ceph-mon[32987]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:30:07.480 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/87021414' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.480 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3550179027' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.480 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.480 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:07 smithi172 ceph-mon[32987]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:30:07.783 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.172 --placement '1;smithi172=iscsi.a' 2024-09-11T08:30:08.032 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:30:08.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:30:08.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:08 smithi043 ceph-mon[29355]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:30:08.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:08 smithi043 ceph-mon[29355]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:30:08.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:08 smithi043 ceph-mon[29355]: pgmap v181: 100 pgs: 2 creating+activating, 13 creating+peering, 61 unknown, 24 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:30:08.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:30:08.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:08 smithi043 ceph-mon[40592]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:30:08.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:08 smithi043 ceph-mon[40592]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:30:08.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:08 smithi043 ceph-mon[40592]: pgmap v181: 100 pgs: 2 creating+activating, 13 creating+peering, 61 unknown, 24 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:30:08.415 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:30:08.415 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:08 smithi172 ceph-mon[32987]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:30:08.415 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:08 smithi172 ceph-mon[32987]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:30:08.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:08 smithi172 ceph-mon[32987]: pgmap v181: 100 pgs: 2 creating+activating, 13 creating+peering, 61 unknown, 24 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:30:09.178 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:09 smithi172 ceph-mon[32987]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:30:09.178 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.178 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2012474113' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.178 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:09 smithi043 ceph-mon[29355]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:30:09.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2012474113' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:09 smithi043 ceph-mon[40592]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:30:09.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2012474113' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 -- 172.21.15.172:0/2067825853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 msgr2=0x7fe9100fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 --2- 172.21.15.172:0/2067825853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 0x7fe9100fe2a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe90c004950 tx=0x7fe90c009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 -- 172.21.15.172:0/2067825853 shutdown_connections 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 --2- 172.21.15.172:0/2067825853 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9100fe7e0 0x7fe9100fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 --2- 172.21.15.172:0/2067825853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9100fb070 0x7fe910104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 --2- 172.21.15.172:0/2067825853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 0x7fe9100fe2a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 -- 172.21.15.172:0/2067825853 >> 172.21.15.172:0/2067825853 conn(0x7fe9100f6690 msgr2=0x7fe9100f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 -- 172.21.15.172:0/2067825853 shutdown_connections 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.661+0000 7fe916e39640 1 -- 172.21.15.172:0/2067825853 wait complete. 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.662+0000 7fe916e39640 1 Processor -- start 2024-09-11T08:30:09.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.662+0000 7fe916e39640 1 -- start start 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.662+0000 7fe916e39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9100fb070 0x7fe91010cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.662+0000 7fe916e39640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 0x7fe91010d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.662+0000 7fe916e39640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9100fe7e0 0x7fe910114320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.662+0000 7fe916e39640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9101148f0 con 0x7fe9100fb070 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.662+0000 7fe916e39640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910114a60 con 0x7fe9100fde90 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.662+0000 7fe916e39640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe910114c20 con 0x7fe9100fe7e0 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 0x7fe91010d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe914bd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9100fb070 0x7fe91010cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 0x7fe91010d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59472/0 (socket says 172.21.15.172:59472) 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 -- 172.21.15.172:0/833382083 learned_addr learned my addr 172.21.15.172:0/833382083 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe9153d6640 1 --2- 172.21.15.172:0/833382083 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9100fe7e0 0x7fe910114320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 -- 172.21.15.172:0/833382083 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9100fe7e0 msgr2=0x7fe910114320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:09.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 --2- 172.21.15.172:0/833382083 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9100fe7e0 0x7fe910114320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 -- 172.21.15.172:0/833382083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9100fb070 msgr2=0x7fe91010cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 --2- 172.21.15.172:0/833382083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9100fb070 0x7fe91010cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 -- 172.21.15.172:0/833382083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c005040 con 0x7fe9100fde90 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe907fff640 1 --2- 172.21.15.172:0/833382083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 0x7fe91010d2c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c860 tx=0x7fe9000043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe9057fa640 1 -- 172.21.15.172:0/833382083 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90000a640 con 0x7fe9100fde90 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe9057fa640 1 -- 172.21.15.172:0/833382083 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fe90000ac10 con 0x7fe9100fde90 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe910114f00 con 0x7fe9100fde90 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe9057fa640 1 -- 172.21.15.172:0/833382083 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900012c80 con 0x7fe9100fde90 2024-09-11T08:30:09.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.663+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9101153c0 con 0x7fe9100fde90 2024-09-11T08:30:09.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.664+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91004bf60 con 0x7fe9100fde90 2024-09-11T08:30:09.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.665+0000 7fe9057fa640 1 -- 172.21.15.172:0/833382083 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7fe90000a7a0 con 0x7fe9100fde90 2024-09-11T08:30:09.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.665+0000 7fe9057fa640 1 --2- 172.21.15.172:0/833382083 >> v2:172.21.15.43:6800/1424089977 conn(0x7fe8f806ee10 0x7fe8f80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:09.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.665+0000 7fe9057fa640 1 -- 172.21.15.172:0/833382083 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5922+0+0 (secure 0 0 0) 0x7fe90008e090 con 0x7fe9100fde90 2024-09-11T08:30:09.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.666+0000 7fe914bd5640 1 --2- 172.21.15.172:0/833382083 >> v2:172.21.15.43:6800/1424089977 conn(0x7fe8f806ee10 0x7fe8f80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:09.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.666+0000 7fe914bd5640 1 --2- 172.21.15.172:0/833382083 >> v2:172.21.15.43:6800/1424089977 conn(0x7fe8f806ee10 0x7fe8f80712c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009040 tx=0x7fe90c009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:09.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.669+0000 7fe9057fa640 1 -- 172.21.15.172:0/833382083 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe90005b110 con 0x7fe9100fde90 2024-09-11T08:30:09.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.835+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.172", "placement": "1;smithi172=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fe91005db50 con 0x7fe8f806ee10 2024-09-11T08:30:09.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.843+0000 7fe9057fa640 1 -- 172.21.15.172:0/833382083 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7fe91005db50 con 0x7fe8f806ee10 2024-09-11T08:30:09.845 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled iscsi.datapool update... 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 >> v2:172.21.15.43:6800/1424089977 conn(0x7fe8f806ee10 msgr2=0x7fe8f80712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 --2- 172.21.15.172:0/833382083 >> v2:172.21.15.43:6800/1424089977 conn(0x7fe8f806ee10 0x7fe8f80712c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009040 tx=0x7fe90c009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 msgr2=0x7fe91010d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 --2- 172.21.15.172:0/833382083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 0x7fe91010d2c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c860 tx=0x7fe9000043c0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 shutdown_connections 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 --2- 172.21.15.172:0/833382083 >> v2:172.21.15.43:6800/1424089977 conn(0x7fe8f806ee10 0x7fe8f80712c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 --2- 172.21.15.172:0/833382083 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9100fe7e0 0x7fe910114320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 --2- 172.21.15.172:0/833382083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9100fde90 0x7fe91010d2c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 --2- 172.21.15.172:0/833382083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9100fb070 0x7fe91010cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.845+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 >> 172.21.15.172:0/833382083 conn(0x7fe9100f6690 msgr2=0x7fe9100f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.846+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 shutdown_connections 2024-09-11T08:30:09.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:09.846+0000 7fe916e39640 1 -- 172.21.15.172:0/833382083 wait complete. 2024-09-11T08:30:10.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:30:10.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:30:10.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[40592]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2012474113' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[40592]: pgmap v184: 132 pgs: 13 creating+peering, 2 creating+activating, 26 unknown, 91 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[29355]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2012474113' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[29355]: pgmap v184: 132 pgs: 13 creating+peering, 2 creating+activating, 26 unknown, 91 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:30:10.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:10 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:10.452 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:30:10.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:30:10.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:10 smithi172 ceph-mon[32987]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:30:10.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2012474113' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:30:10.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:10 smithi172 ceph-mon[32987]: pgmap v184: 132 pgs: 13 creating+peering, 2 creating+activating, 26 unknown, 91 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:30:10.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:10 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:10.480 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T08:30:10.481 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:30:10.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:30:10.511 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:30:10.511 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:30:10.538 DEBUG:teuthology.orchestra.run.smithi172:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T08:30:10.581 INFO:tasks.cephadm:Adding prometheus.a on smithi172 2024-09-11T08:30:10.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi172=a' 2024-09-11T08:30:11.313 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:30:11.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.172", "placement": "1;smithi172=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:11.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: Saving service iscsi.datapool spec with placement smithi172=iscsi.a;count:1 2024-09-11T08:30:11.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[40592]: Deploying daemon iscsi.iscsi.a on smithi172 2024-09-11T08:30:11.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.172", "placement": "1;smithi172=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: Saving service iscsi.datapool spec with placement smithi172=iscsi.a;count:1 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:11.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:11 smithi043 ceph-mon[29355]: Deploying daemon iscsi.iscsi.a on smithi172 2024-09-11T08:30:11.415 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.172", "placement": "1;smithi172=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: Saving service iscsi.datapool spec with placement smithi172=iscsi.a;count:1 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1155898531' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:11.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:11 smithi172 ceph-mon[32987]: Deploying daemon iscsi.iscsi.a on smithi172 2024-09-11T08:30:12.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:12 smithi172 ceph-mon[32987]: pgmap v186: 132 pgs: 13 creating+peering, 13 unknown, 106 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.4 KiB/s rd, 1.6 KiB/s wr, 19 op/s 2024-09-11T08:30:12.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:12 smithi043 ceph-mon[40592]: pgmap v186: 132 pgs: 13 creating+peering, 13 unknown, 106 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.4 KiB/s rd, 1.6 KiB/s wr, 19 op/s 2024-09-11T08:30:12.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:12 smithi043 ceph-mon[29355]: pgmap v186: 132 pgs: 13 creating+peering, 13 unknown, 106 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.4 KiB/s rd, 1.6 KiB/s wr, 19 op/s 2024-09-11T08:30:13.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 -- 172.21.15.172:0/3722196740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fb6d0 msgr2=0x7fd2f40fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 --2- 172.21.15.172:0/3722196740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fb6d0 0x7fd2f40fbae0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0004950 tx=0x7fd2f0009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 -- 172.21.15.172:0/3722196740 shutdown_connections 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 --2- 172.21.15.172:0/3722196740 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2f40fc900 0x7fd2f40fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 --2- 172.21.15.172:0/3722196740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f40fd2b0 0x7fd2f4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 --2- 172.21.15.172:0/3722196740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fb6d0 0x7fd2f40fbae0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 -- 172.21.15.172:0/3722196740 >> 172.21.15.172:0/3722196740 conn(0x7fd2f40f6e90 msgr2=0x7fd2f40f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 -- 172.21.15.172:0/3722196740 shutdown_connections 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.349+0000 7fd2fb281640 1 -- 172.21.15.172:0/3722196740 wait complete. 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2fb281640 1 Processor -- start 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2fb281640 1 -- start start 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2fb281640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2f40fb6d0 0x7fd2f40668c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:13.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2fb281640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fc900 0x7fd2f4111ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2fb281640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f40fd2b0 0x7fd2f4114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2fb281640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f41147f0 con 0x7fd2f40fd2b0 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2fb281640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4114960 con 0x7fd2f40fc900 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2fb281640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4114b20 con 0x7fd2f40fb6d0 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f881c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fc900 0x7fd2f4111ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f881c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fc900 0x7fd2f4111ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59500/0 (socket says 172.21.15.172:59500) 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f881c640 1 -- 172.21.15.172:0/4152671719 learned_addr learned my addr 172.21.15.172:0/4152671719 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f901d640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2f40fb6d0 0x7fd2f40668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f981e640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f40fd2b0 0x7fd2f4114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f881c640 1 -- 172.21.15.172:0/4152671719 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2f40fb6d0 msgr2=0x7fd2f40668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f881c640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2f40fb6d0 0x7fd2f40668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f881c640 1 -- 172.21.15.172:0/4152671719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f40fd2b0 msgr2=0x7fd2f4114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f881c640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f40fd2b0 0x7fd2f4114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.350+0000 7fd2f881c640 1 -- 172.21.15.172:0/4152671719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f0005040 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.351+0000 7fd2f881c640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fc900 0x7fd2f4111ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4002ad0 tx=0x7fd2e4004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.351+0000 7fd2e9ffb640 1 -- 172.21.15.172:0/4152671719 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4007c30 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.351+0000 7fd2e9ffb640 1 -- 172.21.15.172:0/4152671719 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd2e400f040 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.351+0000 7fd2e9ffb640 1 -- 172.21.15.172:0/4152671719 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4009340 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.351+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f4114e00 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.351+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f41152d0 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.352+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f404bf60 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.352+0000 7fd2e9ffb640 1 -- 172.21.15.172:0/4152671719 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) v1 ==== 90745+0+0 (secure 0 0 0) 0x7fd2e4009770 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.353+0000 7fd2e9ffb640 1 --2- 172.21.15.172:0/4152671719 >> v2:172.21.15.43:6800/1424089977 conn(0x7fd2e006ee90 0x7fd2e0071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.353+0000 7fd2e9ffb640 1 -- 172.21.15.172:0/4152671719 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (secure 0 0 0) 0x7fd2e408e080 con 0x7fd2f40fc900 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.353+0000 7fd2f901d640 1 --2- 172.21.15.172:0/4152671719 >> v2:172.21.15.43:6800/1424089977 conn(0x7fd2e006ee90 0x7fd2e0071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:13.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.354+0000 7fd2f901d640 1 --2- 172.21.15.172:0/4152671719 >> v2:172.21.15.43:6800/1424089977 conn(0x7fd2e006ee90 0x7fd2e0071340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd2f002a570 tx=0x7fd2f0035040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:13.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.357+0000 7fd2e9ffb640 1 -- 172.21.15.172:0/4152671719 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd2e405b040 con 0x7fd2f40fc900 2024-09-11T08:30:13.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.538+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi172=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2f4106730 con 0x7fd2e006ee90 2024-09-11T08:30:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.545+0000 7fd2e9ffb640 1 -- 172.21.15.172:0/4152671719 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fd2f4106730 con 0x7fd2e006ee90 2024-09-11T08:30:13.545 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled prometheus update... 2024-09-11T08:30:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 >> v2:172.21.15.43:6800/1424089977 conn(0x7fd2e006ee90 msgr2=0x7fd2e0071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 --2- 172.21.15.172:0/4152671719 >> v2:172.21.15.43:6800/1424089977 conn(0x7fd2e006ee90 0x7fd2e0071340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd2f002a570 tx=0x7fd2f0035040 comp rx=0 tx=0).stop 2024-09-11T08:30:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fc900 msgr2=0x7fd2f4111ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fc900 0x7fd2f4111ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4002ad0 tx=0x7fd2e4004600 comp rx=0 tx=0).stop 2024-09-11T08:30:13.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 shutdown_connections 2024-09-11T08:30:13.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 --2- 172.21.15.172:0/4152671719 >> v2:172.21.15.43:6800/1424089977 conn(0x7fd2e006ee90 0x7fd2e0071340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f40fd2b0 0x7fd2f4114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f40fc900 0x7fd2f4111ad0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 --2- 172.21.15.172:0/4152671719 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2f40fb6d0 0x7fd2f40668c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 >> 172.21.15.172:0/4152671719 conn(0x7fd2f40f6e90 msgr2=0x7fd2f40f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 shutdown_connections 2024-09-11T08:30:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:13.547+0000 7fd2fb281640 1 -- 172.21.15.172:0/4152671719 wait complete. 2024-09-11T08:30:14.626 DEBUG:teuthology.orchestra.run.smithi172:prometheus.a> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T08:30:14.630 INFO:tasks.cephadm:Adding node-exporter.a on smithi043 2024-09-11T08:30:14.630 INFO:tasks.cephadm:Adding node-exporter.b on smithi172 2024-09-11T08:30:14.630 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi043=a;smithi172=b' 2024-09-11T08:30:14.802 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:14 smithi172 ceph-mon[32987]: pgmap v187: 132 pgs: 13 creating+peering, 119 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 68 op/s 2024-09-11T08:30:14.802 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:14 smithi172 ceph-mon[32987]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi172=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:14.802 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:14 smithi172 ceph-mon[32987]: Saving service prometheus spec with placement smithi172=a;count:1 2024-09-11T08:30:14.802 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:14 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:14.802 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:14 smithi172 ceph-mon[32987]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:30:14.802 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:14 smithi172 podman[62503]: 2024-09-11 08:30:14.683885937 +0000 UTC m=+0.018891370 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:30:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[40592]: pgmap v187: 132 pgs: 13 creating+peering, 119 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 68 op/s 2024-09-11T08:30:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[40592]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi172=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[40592]: Saving service prometheus spec with placement smithi172=a;count:1 2024-09-11T08:30:14.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:14.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[40592]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:30:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[29355]: pgmap v187: 132 pgs: 13 creating+peering, 119 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 68 op/s 2024-09-11T08:30:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[29355]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi172=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[29355]: Saving service prometheus spec with placement smithi172=a;count:1 2024-09-11T08:30:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:14 smithi043 ceph-mon[29355]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:30:15.163 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:14 smithi172 podman[62503]: 2024-09-11 08:30:14.802712899 +0000 UTC m=+0.137718318 container create e679348ce9fda312d97877b59b332ae7f6afa7c03e70cbc47a922a704e7b3d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T08:30:15.163 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 podman[62503]: 2024-09-11 08:30:15.046918312 +0000 UTC m=+0.381923748 container init e679348ce9fda312d97877b59b332ae7f6afa7c03e70cbc47a922a704e7b3d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T08:30:15.163 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 podman[62503]: 2024-09-11 08:30:15.052175763 +0000 UTC m=+0.387181194 container start e679348ce9fda312d97877b59b332ae7f6afa7c03e70cbc47a922a704e7b3d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T08:30:15.221 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:30:15.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 bash[62503]: e679348ce9fda312d97877b59b332ae7f6afa7c03e70cbc47a922a704e7b3d4e 2024-09-11T08:30:15.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 systemd[1]: Started Ceph iscsi.iscsi.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:30:15.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug Started the configuration object watcher 2024-09-11T08:30:15.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug Checking for config object changes every 1s 2024-09-11T08:30:15.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug Processing osd blocklist entries for this node 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug Reading the configuration object to update local LIO configuration 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug Configuration does not have an entry for this host(smithi172.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: * Environment: production 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: Use a production WSGI server instead. 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: * Debug mode: off 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug * Running on all addresses. 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: * Running on all addresses. 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:30:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug * Running on http://[fe80::ec4:7aff:fe88:7f91]:5000/ (Press CTRL+C to quit) 2024-09-11T08:30:15.733 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: * Running on http://[fe80::ec4:7aff:fe88:7f91]:5000/ (Press CTRL+C to quit) 2024-09-11T08:30:16.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[40592]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[40592]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 5.6 KiB/s wr, 179 op/s 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/3421272898' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[29355]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[29355]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 5.6 KiB/s wr, 179 op/s 2024-09-11T08:30:16.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:16.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/3421272898' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:30:16.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:16 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:16 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:16 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:16 smithi172 ceph-mon[32987]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:30:16.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:16 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:16.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:16 smithi172 ceph-mon[32987]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 5.6 KiB/s wr, 179 op/s 2024-09-11T08:30:16.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:16 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:16.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/3421272898' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:30:16.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 -- 172.21.15.172:0/2241306294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fda70 msgr2=0x7f65700fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:16.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 --2- 172.21.15.172:0/2241306294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fda70 0x7f65700fdee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6564004950 tx=0x7f6564009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:16.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 -- 172.21.15.172:0/2241306294 shutdown_connections 2024-09-11T08:30:16.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 --2- 172.21.15.172:0/2241306294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65700fe420 0x7f6570104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:16.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 --2- 172.21.15.172:0/2241306294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fda70 0x7f65700fdee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:16.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 --2- 172.21.15.172:0/2241306294 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65700fc7a0 0x7f65700fcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:16.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 -- 172.21.15.172:0/2241306294 >> 172.21.15.172:0/2241306294 conn(0x7f65700f7f60 msgr2=0x7f65700fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 -- 172.21.15.172:0/2241306294 shutdown_connections 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.954+0000 7f65769d7640 1 -- 172.21.15.172:0/2241306294 wait complete. 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f65769d7640 1 Processor -- start 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f65769d7640 1 -- start start 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f65769d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65700fc7a0 0x7f657010ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f65769d7640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65700fda70 0x7f657010d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f65769d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fe420 0x7f65701143c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f65769d7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6570114990 con 0x7f65700fc7a0 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f65769d7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6570114b00 con 0x7f65700fe420 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f65769d7640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6570114cc0 con 0x7f65700fda70 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f6574f74640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fe420 0x7f65701143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:16.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f6574f74640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fe420 0x7f65701143c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45812/0 (socket says 172.21.15.172:45812) 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f6574f74640 1 -- 172.21.15.172:0/823243608 learned_addr learned my addr 172.21.15.172:0/823243608 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f656f7fe640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65700fda70 0x7f657010d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.955+0000 7f656ffff640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65700fc7a0 0x7f657010ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f6574f74640 1 -- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65700fda70 msgr2=0x7f657010d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f6574f74640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65700fda70 0x7f657010d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f6574f74640 1 -- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65700fc7a0 msgr2=0x7f657010ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f6574f74640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65700fc7a0 0x7f657010ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f6574f74640 1 -- 172.21.15.172:0/823243608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6564005040 con 0x7f65700fe420 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f656f7fe640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65700fda70 0x7f657010d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f656ffff640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65700fc7a0 0x7f657010ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:30:16.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f6574f74640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fe420 0x7f65701143c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f656800bc30 tx=0x7f6568009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f656cff9640 1 -- 172.21.15.172:0/823243608 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680092e0 con 0x7f65700fe420 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f656cff9640 1 -- 172.21.15.172:0/823243608 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f656800ccd0 con 0x7f65700fe420 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f656cff9640 1 -- 172.21.15.172:0/823243608 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568015d70 con 0x7f65700fe420 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6570114fa0 con 0x7f65700fe420 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.956+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6570115390 con 0x7f65700fe420 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.957+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657004bf60 con 0x7f65700fe420 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.957+0000 7f656cff9640 1 -- 172.21.15.172:0/823243608 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f6568012070 con 0x7f65700fe420 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.958+0000 7f656cff9640 1 --2- 172.21.15.172:0/823243608 >> v2:172.21.15.43:6800/1424089977 conn(0x7f655806ef40 0x7f65580713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.958+0000 7f656cff9640 1 -- 172.21.15.172:0/823243608 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5938+0+0 (secure 0 0 0) 0x7f656808e0c0 con 0x7f65700fe420 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.958+0000 7f656ffff640 1 --2- 172.21.15.172:0/823243608 >> v2:172.21.15.43:6800/1424089977 conn(0x7f655806ef40 0x7f65580713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:16.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.959+0000 7f656ffff640 1 --2- 172.21.15.172:0/823243608 >> v2:172.21.15.43:6800/1424089977 conn(0x7f655806ef40 0x7f65580713f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6560009a80 tx=0x7f6560008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:16.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:16.962+0000 7f656cff9640 1 -- 172.21.15.172:0/823243608 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f656805b090 con 0x7f65700fe420 2024-09-11T08:30:17.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.132+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi172=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f6570107820 con 0x7f655806ef40 2024-09-11T08:30:17.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.139+0000 7f656cff9640 1 -- 172.21.15.172:0/823243608 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f6570107820 con 0x7f655806ef40 2024-09-11T08:30:17.140 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled node-exporter update... 2024-09-11T08:30:17.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.141+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 >> v2:172.21.15.43:6800/1424089977 conn(0x7f655806ef40 msgr2=0x7f65580713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.141+0000 7f65769d7640 1 --2- 172.21.15.172:0/823243608 >> v2:172.21.15.43:6800/1424089977 conn(0x7f655806ef40 0x7f65580713f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6560009a80 tx=0x7f6560008040 comp rx=0 tx=0).stop 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fe420 msgr2=0x7f65701143c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fe420 0x7f65701143c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f656800bc30 tx=0x7f6568009840 comp rx=0 tx=0).stop 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 shutdown_connections 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 --2- 172.21.15.172:0/823243608 >> v2:172.21.15.43:6800/1424089977 conn(0x7f655806ef40 0x7f65580713f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65700fda70 0x7f657010d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700fe420 0x7f65701143c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 --2- 172.21.15.172:0/823243608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65700fc7a0 0x7f657010ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 >> 172.21.15.172:0/823243608 conn(0x7f65700f7f60 msgr2=0x7f65700f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 shutdown_connections 2024-09-11T08:30:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:17.142+0000 7f65769d7640 1 -- 172.21.15.172:0/823243608 wait complete. 2024-09-11T08:30:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:17 smithi172 ceph-mon[32987]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:30:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:17 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:17 smithi043 ceph-mon[40592]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:30:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:17 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:17.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:17 smithi043 ceph-mon[29355]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:30:17.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:17 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:18.279 DEBUG:teuthology.orchestra.run.smithi043:node-exporter.a> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T08:30:18.282 DEBUG:teuthology.orchestra.run.smithi172:node-exporter.b> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T08:30:18.284 INFO:tasks.cephadm:Adding alertmanager.a on smithi043 2024-09-11T08:30:18.284 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi043=a' 2024-09-11T08:30:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:18 smithi043 ceph-mon[40592]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi172=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:18 smithi043 ceph-mon[40592]: Saving service node-exporter spec with placement smithi043=a;smithi172=b;count:2 2024-09-11T08:30:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:18 smithi043 ceph-mon[40592]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-09-11T08:30:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:18 smithi043 ceph-mon[29355]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi172=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:18 smithi043 ceph-mon[29355]: Saving service node-exporter spec with placement smithi043=a;smithi172=b;count:2 2024-09-11T08:30:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:18 smithi043 ceph-mon[29355]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-09-11T08:30:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:18 smithi172 ceph-mon[32987]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi172=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:18.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:18 smithi172 ceph-mon[32987]: Saving service node-exporter spec with placement smithi043=a;smithi172=b;count:2 2024-09-11T08:30:18.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:18 smithi172 ceph-mon[32987]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-09-11T08:30:19.282 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:30:20.417 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:20 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.417 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:20 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.417 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:20 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:20.417 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:20 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:20.417 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:20 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.417 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:20 smithi172 ceph-mon[32987]: Deploying daemon prometheus.a on smithi172 2024-09-11T08:30:20.418 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:20 smithi172 ceph-mon[32987]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 50 KiB/s rd, 3.8 KiB/s wr, 121 op/s 2024-09-11T08:30:20.418 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:20 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[29355]: Deploying daemon prometheus.a on smithi172 2024-09-11T08:30:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[29355]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 50 KiB/s rd, 3.8 KiB/s wr, 121 op/s 2024-09-11T08:30:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:20.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:20.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[40592]: Deploying daemon prometheus.a on smithi172 2024-09-11T08:30:20.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[40592]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 50 KiB/s rd, 3.8 KiB/s wr, 121 op/s 2024-09-11T08:30:20.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:20 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:20.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.899+0000 7fb87189f640 1 -- 172.21.15.172:0/3383303791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fb3a0 msgr2=0x7fb86c0fb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:20.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.899+0000 7fb87189f640 1 --2- 172.21.15.172:0/3383303791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fb3a0 0x7fb86c0fb810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb860004950 tx=0x7fb860009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:20.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 -- 172.21.15.172:0/3383303791 shutdown_connections 2024-09-11T08:30:20.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 --2- 172.21.15.172:0/3383303791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0fbd50 0x7fb86c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:20.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 --2- 172.21.15.172:0/3383303791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fb3a0 0x7fb86c0fb810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:20.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 --2- 172.21.15.172:0/3383303791 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb86c0fcd90 0x7fb86c0fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:20.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 -- 172.21.15.172:0/3383303791 >> 172.21.15.172:0/3383303791 conn(0x7fb86c0f6e70 msgr2=0x7fb86c0f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:20.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 -- 172.21.15.172:0/3383303791 shutdown_connections 2024-09-11T08:30:20.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 -- 172.21.15.172:0/3383303791 wait complete. 2024-09-11T08:30:20.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 Processor -- start 2024-09-11T08:30:20.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 -- start start 2024-09-11T08:30:20.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb86c0fb3a0 0x7fb86c106180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:20.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0fbd50 0x7fb86c104670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:20.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fcd90 0x7fb86c104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:20.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c105480 con 0x7fb86c0fbd50 2024-09-11T08:30:20.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c117020 con 0x7fb86c0fcd90 2024-09-11T08:30:20.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.900+0000 7fb87189f640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c117160 con 0x7fb86c0fb3a0 2024-09-11T08:30:20.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fcd90 0x7fb86c104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:20.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fcd90 0x7fb86c104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45826/0 (socket says 172.21.15.172:45826) 2024-09-11T08:30:20.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 -- 172.21.15.172:0/1654794679 learned_addr learned my addr 172.21.15.172:0/1654794679 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:30:20.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86a7fc640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0fbd50 0x7fb86c104670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:20.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86affd640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb86c0fb3a0 0x7fb86c106180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:20.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 -- 172.21.15.172:0/1654794679 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb86c0fb3a0 msgr2=0x7fb86c106180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:20.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb86c0fb3a0 0x7fb86c106180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:20.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 -- 172.21.15.172:0/1654794679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0fbd50 msgr2=0x7fb86c104670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0fbd50 0x7fb86c104670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 -- 172.21.15.172:0/1654794679 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860005040 con 0x7fb86c0fcd90 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb86b7fe640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fcd90 0x7fb86c104cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb86400b5e0 tx=0x7fb8640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb8537fe640 1 -- 172.21.15.172:0/1654794679 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86400f590 con 0x7fb86c0fcd90 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb8537fe640 1 -- 172.21.15.172:0/1654794679 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fb86400fb60 con 0x7fb86c0fcd90 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c117440 con 0x7fb86c0fcd90 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb8537fe640 1 -- 172.21.15.172:0/1654794679 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864010b40 con 0x7fb86c0fcd90 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.901+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c117910 con 0x7fb86c0fcd90 2024-09-11T08:30:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.902+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c107ea0 con 0x7fb86c0fcd90 2024-09-11T08:30:20.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.903+0000 7fb8537fe640 1 -- 172.21.15.172:0/1654794679 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) v1 ==== 90796+0+0 (secure 0 0 0) 0x7fb86400f6f0 con 0x7fb86c0fcd90 2024-09-11T08:30:20.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.903+0000 7fb8537fe640 1 --2- 172.21.15.172:0/1654794679 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ee70 0x7fb854071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:20.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.904+0000 7fb8537fe640 1 -- 172.21.15.172:0/1654794679 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5938+0+0 (secure 0 0 0) 0x7fb86408e0d0 con 0x7fb86c0fcd90 2024-09-11T08:30:20.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.904+0000 7fb86affd640 1 --2- 172.21.15.172:0/1654794679 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ee70 0x7fb854071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:20.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.904+0000 7fb86affd640 1 --2- 172.21.15.172:0/1654794679 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ee70 0x7fb854071320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb86c105f30 tx=0x7fb85c009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:20.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:20.908+0000 7fb8537fe640 1 -- 172.21.15.172:0/1654794679 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb86405b1d0 con 0x7fb86c0fcd90 2024-09-11T08:30:21.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.082+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb86c04b790 con 0x7fb85406ee70 2024-09-11T08:30:21.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.089+0000 7fb8537fe640 1 -- 172.21.15.172:0/1654794679 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fb86c04b790 con 0x7fb85406ee70 2024-09-11T08:30:21.089 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled alertmanager update... 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.090+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ee70 msgr2=0x7fb854071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.090+0000 7fb87189f640 1 --2- 172.21.15.172:0/1654794679 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ee70 0x7fb854071320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb86c105f30 tx=0x7fb85c009780 comp rx=0 tx=0).stop 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.090+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fcd90 msgr2=0x7fb86c104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fcd90 0x7fb86c104cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb86400b5e0 tx=0x7fb8640091a0 comp rx=0 tx=0).stop 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 shutdown_connections 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 --2- 172.21.15.172:0/1654794679 >> v2:172.21.15.43:6800/1424089977 conn(0x7fb85406ee70 0x7fb854071320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c0fcd90 0x7fb86c104cf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0fbd50 0x7fb86c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:21.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 --2- 172.21.15.172:0/1654794679 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb86c0fb3a0 0x7fb86c106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:21.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 >> 172.21.15.172:0/1654794679 conn(0x7fb86c0f6e70 msgr2=0x7fb86c0f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:21.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 shutdown_connections 2024-09-11T08:30:21.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:21.091+0000 7fb87189f640 1 -- 172.21.15.172:0/1654794679 wait complete. 2024-09-11T08:30:22.058 DEBUG:teuthology.orchestra.run.smithi043:alertmanager.a> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T08:30:22.061 INFO:tasks.cephadm:Adding grafana.a on smithi172 2024-09-11T08:30:22.062 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi172=a' 2024-09-11T08:30:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:30:22.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:22 smithi043 ceph-mon[40592]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:22.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:22 smithi043 ceph-mon[40592]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-11T08:30:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:22 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:22 smithi043 ceph-mon[40592]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 3.3 KiB/s wr, 106 op/s 2024-09-11T08:30:22.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:22 smithi043 ceph-mon[29355]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:22.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:22 smithi043 ceph-mon[29355]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-11T08:30:22.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:22 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:22.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:22 smithi043 ceph-mon[29355]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 3.3 KiB/s wr, 106 op/s 2024-09-11T08:30:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:22 smithi172 ceph-mon[32987]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:22 smithi172 ceph-mon[32987]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-11T08:30:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:22 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:22 smithi172 ceph-mon[32987]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 3.3 KiB/s wr, 106 op/s 2024-09-11T08:30:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:24 smithi172 ceph-mon[32987]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 73 op/s 2024-09-11T08:30:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:24 smithi043 ceph-mon[40592]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 73 op/s 2024-09-11T08:30:24.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:24 smithi043 ceph-mon[29355]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 73 op/s 2024-09-11T08:30:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 -- 172.21.15.172:0/1695551408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fd2b0 msgr2=0x7f5a6c103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 --2- 172.21.15.172:0/1695551408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fd2b0 0x7f5a6c103b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009cd0 tx=0x7f5a64004d10 comp rx=0 tx=0).stop 2024-09-11T08:30:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 -- 172.21.15.172:0/1695551408 shutdown_connections 2024-09-11T08:30:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 --2- 172.21.15.172:0/1695551408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fd2b0 0x7f5a6c103b50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 --2- 172.21.15.172:0/1695551408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c0fc900 0x7f5a6c0fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 --2- 172.21.15.172:0/1695551408 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a6c0fb6d0 0x7f5a6c0fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 -- 172.21.15.172:0/1695551408 >> 172.21.15.172:0/1695551408 conn(0x7f5a6c0f6e90 msgr2=0x7f5a6c0f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 -- 172.21.15.172:0/1695551408 shutdown_connections 2024-09-11T08:30:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.838+0000 7f5a71920640 1 -- 172.21.15.172:0/1695551408 wait complete. 2024-09-11T08:30:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a71920640 1 Processor -- start 2024-09-11T08:30:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a71920640 1 -- start start 2024-09-11T08:30:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a71920640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c0fb6d0 0x7f5a6c10b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a71920640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fc900 0x7f5a6c10be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a71920640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a6c0fd2b0 0x7f5a6c112ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a71920640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c1134a0 con 0x7f5a6c0fb6d0 2024-09-11T08:30:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a71920640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c113610 con 0x7f5a6c0fc900 2024-09-11T08:30:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a71920640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c1137d0 con 0x7f5a6c0fd2b0 2024-09-11T08:30:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a6a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fc900 0x7f5a6c10be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a6a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fc900 0x7f5a6c10be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45848/0 (socket says 172.21.15.172:45848) 2024-09-11T08:30:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.839+0000 7f5a6a7fc640 1 -- 172.21.15.172:0/824696039 learned_addr learned my addr 172.21.15.172:0/824696039 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:30:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a6affd640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c0fb6d0 0x7f5a6c10b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a6b7fe640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a6c0fd2b0 0x7f5a6c112ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:24.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a6a7fc640 1 -- 172.21.15.172:0/824696039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a6c0fd2b0 msgr2=0x7f5a6c112ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:24.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a6a7fc640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a6c0fd2b0 0x7f5a6c112ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:24.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a6a7fc640 1 -- 172.21.15.172:0/824696039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c0fb6d0 msgr2=0x7f5a6c10b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:24.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a6a7fc640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c0fb6d0 0x7f5a6c10b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:24.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a6a7fc640 1 -- 172.21.15.172:0/824696039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640098b0 con 0x7f5a6c0fc900 2024-09-11T08:30:24.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a6a7fc640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fc900 0x7f5a6c10be70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000c890 tx=0x7f5a600043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:24.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a537fe640 1 -- 172.21.15.172:0/824696039 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000ce20 con 0x7f5a6c0fc900 2024-09-11T08:30:24.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a537fe640 1 -- 172.21.15.172:0/824696039 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f5a6000acc0 con 0x7f5a6c0fc900 2024-09-11T08:30:24.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c113ab0 con 0x7f5a6c0fc900 2024-09-11T08:30:24.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a537fe640 1 -- 172.21.15.172:0/824696039 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60012ca0 con 0x7f5a6c0fc900 2024-09-11T08:30:24.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.840+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c113fd0 con 0x7f5a6c0fc900 2024-09-11T08:30:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.841+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c04bf60 con 0x7f5a6c0fc900 2024-09-11T08:30:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.842+0000 7f5a537fe640 1 -- 172.21.15.172:0/824696039 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f5a6000a850 con 0x7f5a6c0fc900 2024-09-11T08:30:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.842+0000 7f5a537fe640 1 --2- 172.21.15.172:0/824696039 >> v2:172.21.15.43:6800/1424089977 conn(0x7f5a5406ef40 0x7f5a540713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.842+0000 7f5a537fe640 1 -- 172.21.15.172:0/824696039 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5938+0+0 (secure 0 0 0) 0x7f5a6008e5b0 con 0x7f5a6c0fc900 2024-09-11T08:30:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.842+0000 7f5a6affd640 1 --2- 172.21.15.172:0/824696039 >> v2:172.21.15.43:6800/1424089977 conn(0x7f5a5406ef40 0x7f5a540713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.843+0000 7f5a6affd640 1 --2- 172.21.15.172:0/824696039 >> v2:172.21.15.43:6800/1424089977 conn(0x7f5a5406ef40 0x7f5a540713f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0fc760 tx=0x7f5a5c009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:24.846+0000 7f5a537fe640 1 -- 172.21.15.172:0/824696039 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5a6005b630 con 0x7f5a6c0fc900 2024-09-11T08:30:25.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.012+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 --> v2:172.21.15.43:6800/1424089977 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi172=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a6c106720 con 0x7f5a5406ef40 2024-09-11T08:30:25.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.019+0000 7f5a537fe640 1 -- 172.21.15.172:0/824696039 <== mgr.14150 v2:172.21.15.43:6800/1424089977 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f5a6c106720 con 0x7f5a5406ef40 2024-09-11T08:30:25.019 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled grafana update... 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 >> v2:172.21.15.43:6800/1424089977 conn(0x7f5a5406ef40 msgr2=0x7f5a540713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 --2- 172.21.15.172:0/824696039 >> v2:172.21.15.43:6800/1424089977 conn(0x7f5a5406ef40 0x7f5a540713f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0fc760 tx=0x7f5a5c009040 comp rx=0 tx=0).stop 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fc900 msgr2=0x7f5a6c10be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fc900 0x7f5a6c10be70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000c890 tx=0x7f5a600043c0 comp rx=0 tx=0).stop 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 shutdown_connections 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 --2- 172.21.15.172:0/824696039 >> v2:172.21.15.43:6800/1424089977 conn(0x7f5a5406ef40 0x7f5a540713f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a6c0fd2b0 0x7f5a6c112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c0fc900 0x7f5a6c10be70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 --2- 172.21.15.172:0/824696039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c0fb6d0 0x7f5a6c10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:25.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 >> 172.21.15.172:0/824696039 conn(0x7f5a6c0f6e90 msgr2=0x7f5a6c0f7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:25.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 shutdown_connections 2024-09-11T08:30:25.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:25.021+0000 7f5a71920640 1 -- 172.21.15.172:0/824696039 wait complete. 2024-09-11T08:30:25.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:30:25.913 DEBUG:teuthology.orchestra.run.smithi172:grafana.a> sudo journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T08:30:25.915 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T08:30:25.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:30:26.183 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:26.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:26 smithi172 ceph-mon[32987]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi172=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:26.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:26 smithi172 ceph-mon[32987]: Saving service grafana spec with placement smithi172=a;count:1 2024-09-11T08:30:26.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:26 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:26.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:26 smithi172 ceph-mon[32987]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 272 B/s wr, 7 op/s 2024-09-11T08:30:26.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:26 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:26.304 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[40592]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi172=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:26.304 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[40592]: Saving service grafana spec with placement smithi172=a;count:1 2024-09-11T08:30:26.304 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:26.304 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[40592]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 272 B/s wr, 7 op/s 2024-09-11T08:30:26.304 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:26.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[29355]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi172=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:30:26.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[29355]: Saving service grafana spec with placement smithi172=a;count:1 2024-09-11T08:30:26.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:26.305 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[29355]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 272 B/s wr, 7 op/s 2024-09-11T08:30:26.305 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:26 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:28.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.021+0000 7f061c0c6640 1 -- 172.21.15.43:0/3867822915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 msgr2=0x7f06140faa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:28.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.021+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3867822915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 0x7f06140faa10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0604004890 tx=0x7f0604009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.021+0000 7f061c0c6640 1 -- 172.21.15.43:0/3867822915 shutdown_connections 2024-09-11T08:30:28.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.021+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3867822915 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06140fb030 0x7f06140fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.021+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3867822915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 0x7f06140faa10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.021+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3867822915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06140fdf60 0x7f06140fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.021+0000 7f061c0c6640 1 -- 172.21.15.43:0/3867822915 >> 172.21.15.43:0/3867822915 conn(0x7f06140f66a0 msgr2=0x7f06140f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 -- 172.21.15.43:0/3867822915 shutdown_connections 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 -- 172.21.15.43:0/3867822915 wait complete. 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 Processor -- start 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 -- start start 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06140fb030 0x7f06140fd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06140fdf60 0x7f06140fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 0x7f06140fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06140fc820 con 0x7f06140fe8b0 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06140fc990 con 0x7f06140fb030 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061c0c6640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0614116ee0 con 0x7f06140fdf60 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f0619661640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06140fdf60 0x7f06140fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:28.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f0619661640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06140fdf60 0x7f06140fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:39740/0 (socket says 172.21.15.43:39740) 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f061a663640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 0x7f06140fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f0619661640 1 -- 172.21.15.43:0/3420233561 learned_addr learned my addr 172.21.15.43:0/3420233561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.022+0000 7f0619e62640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06140fb030 0x7f06140fd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f0619661640 1 -- 172.21.15.43:0/3420233561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06140fb030 msgr2=0x7f06140fd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f0619661640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06140fb030 0x7f06140fd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f0619661640 1 -- 172.21.15.43:0/3420233561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 msgr2=0x7f06140fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f0619661640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 0x7f06140fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f0619661640 1 -- 172.21.15.43:0/3420233561 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0098b0 con 0x7f06140fdf60 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f061a663640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 0x7f06140fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f0619661640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06140fdf60 0x7f06140fbc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0604004860 tx=0x7f0604009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f060a7fc640 1 -- 172.21.15.43:0/3420233561 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604007d00 con 0x7f06140fdf60 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0604005040 con 0x7f06140fdf60 2024-09-11T08:30:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f060a7fc640 1 -- 172.21.15.43:0/3420233561 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f06040090d0 con 0x7f06140fdf60 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f060a7fc640 1 -- 172.21.15.43:0/3420233561 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604033690 con 0x7f06140fdf60 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.023+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0614117490 con 0x7f06140fdf60 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.024+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0614107ed0 con 0x7f06140fdf60 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.025+0000 7f060a7fc640 1 -- 172.21.15.43:0/3420233561 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 16) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f0604030020 con 0x7f06140fdf60 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.025+0000 7f060a7fc640 1 --2- 172.21.15.43:0/3420233561 >> v2:172.21.15.43:6800/1424089977 conn(0x7f060006ee70 0x7f0600071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.025+0000 7f060a7fc640 1 -- 172.21.15.43:0/3420233561 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5938+0+0 (secure 0 0 0) 0x7f06040ae720 con 0x7f06140fdf60 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.025+0000 7f0619e62640 1 --2- 172.21.15.43:0/3420233561 >> v2:172.21.15.43:6800/1424089977 conn(0x7f060006ee70 0x7f0600071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.026+0000 7f0619e62640 1 --2- 172.21.15.43:0/3420233561 >> v2:172.21.15.43:6800/1424089977 conn(0x7f060006ee70 0x7f0600071320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0610004e20 tx=0x7f061000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.030+0000 7f060a7fc640 1 -- 172.21.15.43:0/3420233561 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f060407b860 con 0x7f06140fdf60 2024-09-11T08:30:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.257+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f06141076e0 con 0x7f06140fdf60 2024-09-11T08:30:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.261+0000 7f060a7fc640 1 -- 172.21.15.43:0/3420233561 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f060407b200 con 0x7f06140fdf60 2024-09-11T08:30:28.261 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-11T08:30:28.261 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAkVeFmLA5uDxAAePHIMEkT2NfBzj9Iy6CRzw== 2024-09-11T08:30:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 >> v2:172.21.15.43:6800/1424089977 conn(0x7f060006ee70 msgr2=0x7f0600071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3420233561 >> v2:172.21.15.43:6800/1424089977 conn(0x7f060006ee70 0x7f0600071320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0610004e20 tx=0x7f061000b040 comp rx=0 tx=0).stop 2024-09-11T08:30:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06140fdf60 msgr2=0x7f06140fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06140fdf60 0x7f06140fbc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0604004860 tx=0x7f0604009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 shutdown_connections 2024-09-11T08:30:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3420233561 >> v2:172.21.15.43:6800/1424089977 conn(0x7f060006ee70 0x7f0600071320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06140fdf60 0x7f06140fbc40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06140fe8b0 0x7f06140fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 --2- 172.21.15.43:0/3420233561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06140fb030 0x7f06140fd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 >> 172.21.15.43:0/3420233561 conn(0x7f06140f66a0 msgr2=0x7f06140f89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 shutdown_connections 2024-09-11T08:30:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:28.263+0000 7f061c0c6640 1 -- 172.21.15.43:0/3420233561 wait complete. 2024-09-11T08:30:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:28 smithi172 ceph-mon[32987]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3420233561' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:30:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:28 smithi043 ceph-mon[40592]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:28.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3420233561' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:28.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:28.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:30:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:28 smithi043 ceph-mon[29355]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3420233561' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:30:28.939 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:30:28.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T08:30:28.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T08:30:28.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:30:30.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:30 smithi172 ceph-mon[32987]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:30 smithi043 ceph-mon[40592]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:30.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:30 smithi043 ceph-mon[29355]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:30:32.509 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:32 smithi172 ceph-mon[32987]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:32 smithi043 ceph-mon[40592]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:32 smithi043 ceph-mon[29355]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:30:33.213 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:33 smithi172 systemd[1]: Starting Ceph prometheus.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:30:33.982 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:33 smithi172 podman[64370]: 2024-09-11 08:30:33.557034389 +0000 UTC m=+0.018998421 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:30:34.676 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:34 smithi172 ceph-mon[32987]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:30:34.697 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:34 smithi043 ceph-mon[40592]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:30:34.697 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:34 smithi043 ceph-mon[29355]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:30:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 -- 172.21.15.172:0/2121558390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 msgr2=0x7f4f1c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 --2- 172.21.15.172:0/2121558390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 0x7f4f1c108940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f10004950 tx=0x7f4f10009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 -- 172.21.15.172:0/2121558390 shutdown_connections 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 --2- 172.21.15.172:0/2121558390 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c108e80 0x7f4f1c10f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 --2- 172.21.15.172:0/2121558390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 0x7f4f1c108940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 --2- 172.21.15.172:0/2121558390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c107200 0x7f4f1c107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 -- 172.21.15.172:0/2121558390 >> 172.21.15.172:0/2121558390 conn(0x7f4f1c0677d0 msgr2=0x7f4f1c067bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 -- 172.21.15.172:0/2121558390 shutdown_connections 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.675+0000 7f4f22862640 1 -- 172.21.15.172:0/2121558390 wait complete. 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f22862640 1 Processor -- start 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f22862640 1 -- start start 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f22862640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c107200 0x7f4f1c117670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f22862640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 0x7f4f1c117bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f22862640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c108e80 0x7f4f1c11ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f22862640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c11f1e0 con 0x7f4f1c108e80 2024-09-11T08:30:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f22862640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c11f350 con 0x7f4f1c1084d0 2024-09-11T08:30:35.246 INFO:teuthology.orchestra.run.smithi172.stdout:[client.1] 2024-09-11T08:30:35.246 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQAqVeFmYxGbNRAAXb4l6A99uyaPQuedbakYrQ== 2024-09-11T08:30:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f22862640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c11f510 con 0x7f4f1c107200 2024-09-11T08:30:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f1b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 0x7f4f1c117bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f1b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 0x7f4f1c117bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47374/0 (socket says 172.21.15.172:47374) 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f1b7fe640 1 -- 172.21.15.172:0/1068257423 learned_addr learned my addr 172.21.15.172:0/1068257423 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f1bfff640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c107200 0x7f4f1c117670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f20dff640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c108e80 0x7f4f1c11ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f1b7fe640 1 -- 172.21.15.172:0/1068257423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c107200 msgr2=0x7f4f1c117670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f1b7fe640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c107200 0x7f4f1c117670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.676+0000 7f4f1b7fe640 1 -- 172.21.15.172:0/1068257423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c108e80 msgr2=0x7f4f1c11ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.677+0000 7f4f1b7fe640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c108e80 0x7f4f1c11ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.677+0000 7f4f1b7fe640 1 -- 172.21.15.172:0/1068257423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10005040 con 0x7f4f1c1084d0 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.677+0000 7f4f1b7fe640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 0x7f4f1c117bb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f100043d0 tx=0x7f4f10035040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.677+0000 7f4f18ff9640 1 -- 172.21.15.172:0/1068257423 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1002ab70 con 0x7f4f1c1084d0 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.677+0000 7f4f18ff9640 1 -- 172.21.15.172:0/1068257423 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f4f1002acd0 con 0x7f4f1c1084d0 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.677+0000 7f4f18ff9640 1 -- 172.21.15.172:0/1068257423 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10033670 con 0x7f4f1c1084d0 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.677+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c11f790 con 0x7f4f1c1084d0 2024-09-11T08:30:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.677+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c11fc90 con 0x7f4f1c1084d0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.678+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c04bf60 con 0x7f4f1c1084d0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.679+0000 7f4f18ff9640 1 -- 172.21.15.172:0/1068257423 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f4f10030020 con 0x7f4f1c1084d0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.679+0000 7f4f18ff9640 1 --2- 172.21.15.172:0/1068257423 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4f0406eea0 0x7f4f04071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.679+0000 7f4f18ff9640 1 -- 172.21.15.172:0/1068257423 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5938+0+0 (secure 0 0 0) 0x7f4f100ae870 con 0x7f4f1c1084d0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.679+0000 7f4f1bfff640 1 --2- 172.21.15.172:0/1068257423 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4f0406eea0 0x7f4f04071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.680+0000 7f4f1bfff640 1 --2- 172.21.15.172:0/1068257423 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4f0406eea0 0x7f4f04071350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009fd0 tx=0x7f4f0c009e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.683+0000 7f4f18ff9640 1 -- 172.21.15.172:0/1068257423 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f4f1007b840 con 0x7f4f1c1084d0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.897+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f4f1c112180 con 0x7f4f1c1084d0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.904+0000 7f4f18ff9640 1 -- 172.21.15.172:0/1068257423 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f4f1007b1e0 con 0x7f4f1c1084d0 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4f0406eea0 msgr2=0x7f4f04071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 --2- 172.21.15.172:0/1068257423 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4f0406eea0 0x7f4f04071350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009fd0 tx=0x7f4f0c009e50 comp rx=0 tx=0).stop 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 msgr2=0x7f4f1c117bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 0x7f4f1c117bb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f100043d0 tx=0x7f4f10035040 comp rx=0 tx=0).stop 2024-09-11T08:30:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 shutdown_connections 2024-09-11T08:30:35.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 --2- 172.21.15.172:0/1068257423 >> v2:172.21.15.43:6800/1424089977 conn(0x7f4f0406eea0 0x7f4f04071350 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c108e80 0x7f4f1c11ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f1c1084d0 0x7f4f1c117bb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 --2- 172.21.15.172:0/1068257423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c107200 0x7f4f1c117670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:35.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.906+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 >> 172.21.15.172:0/1068257423 conn(0x7f4f1c0677d0 msgr2=0x7f4f1c10b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:35.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.907+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 shutdown_connections 2024-09-11T08:30:35.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T08:30:34.907+0000 7f4f22862640 1 -- 172.21.15.172:0/1068257423 wait complete. 2024-09-11T08:30:35.645 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.172:0/1068257423' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:35.646 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:35.646 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:30:35.646 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:30:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.172:0/1068257423' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:30:35.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.172:0/1068257423' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:35.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:30:35.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:30:35.981 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:35 smithi172 podman[64370]: 2024-09-11 08:30:35.64620458 +0000 UTC m=+2.108168643 container create f29653c1e60b9efeba0ef612977464615924215eaa826ae210c4b8968181b69e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:30:36.213 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T08:30:36.213 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T08:30:36.213 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T08:30:36.257 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T08:30:36.257 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T08:30:36.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T08:30:36.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-mon[32987]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:30:36.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 podman[64370]: 2024-09-11 08:30:36.326431809 +0000 UTC m=+2.788395828 container init f29653c1e60b9efeba0ef612977464615924215eaa826ae210c4b8968181b69e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 podman[64370]: 2024-09-11 08:30:36.332594115 +0000 UTC m=+2.794558136 container start f29653c1e60b9efeba0ef612977464615924215eaa826ae210c4b8968181b69e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.358Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.358Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.358Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi172 (none))" 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.358Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.358Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.359Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:30:36.482 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.359Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.360Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.360Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.362Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.362Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.863µs 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.362Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.363Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.363Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.169µs wal_replay_duration=1.302534ms wbl_replay_duration=143ns total_replay_duration=1.337594ms 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.364Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.364Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.364Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.379Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.734124ms db_storage=754ns remote_storage=1.098µs web_handler=315ns query_engine=533ns scrape=508.139µs scrape_sd=21.68µs notify=1.054µs notify_sd=2.26µs rules=14.005983ms tracing=5.557µs 2024-09-11T08:30:36.483 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.379Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:30:36.484 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:30:36.379Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:30:36.484 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 bash[64370]: f29653c1e60b9efeba0ef612977464615924215eaa826ae210c4b8968181b69e 2024-09-11T08:30:36.484 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:30:36 smithi172 systemd[1]: Started Ceph prometheus.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:30:36.525 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:36 smithi043 ceph-mon[40592]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:30:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:36 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:36.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:36 smithi043 ceph-mon[29355]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:30:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:36 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:30:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:30:37.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ignoring --setuser ceph since I am not root 2024-09-11T08:30:37.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ignoring --setgroup ceph since I am not root 2024-09-11T08:30:37.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-mgr[29632]: -- 172.21.15.43:0/1267918750 <== mon.2 v2:172.21.15.43:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c23659a340 con 0x55c2365a6c00 2024-09-11T08:30:37.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:37.614+0000 7fa3ae01bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:30:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' 2024-09-11T08:30:37.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:30:37.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ignoring --setuser ceph since I am not root 2024-09-11T08:30:37.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ignoring --setgroup ceph since I am not root 2024-09-11T08:30:37.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-mgr[35100]: -- 172.21.15.172:0/938471466 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e274ea4340 con 0x55e274eb0c00 2024-09-11T08:30:37.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:37.611+0000 7ff3bd551ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:30:37.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:37.666+0000 7ff3bd551ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:30:37.943 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:37.671+0000 7fa3ae01bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:30:37.943 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:37.739+0000 7fa3ae01bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:30:37.943 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:37.872+0000 7fa3ae01bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:30:37.996 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:37.732+0000 7ff3bd551ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:30:37.996 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:37.857+0000 7ff3bd551ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:30:37.996 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:37 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:37.996+0000 7ff3bd551ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:30:38.198 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:38.016+0000 7fa3ae01bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:30:38.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 -- 172.21.15.43:0/1402900656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081027e0 msgr2=0x7ff708102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 --2- 172.21.15.43:0/1402900656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081027e0 0x7ff708102bb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff704004950 tx=0x7ff704009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 -- 172.21.15.43:0/1402900656 shutdown_connections 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 --2- 172.21.15.43:0/1402900656 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff708103180 0x7ff7080fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 --2- 172.21.15.43:0/1402900656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7080fb4a0 0x7ff7081095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 --2- 172.21.15.43:0/1402900656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081027e0 0x7ff708102bb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 -- 172.21.15.43:0/1402900656 >> 172.21.15.43:0/1402900656 conn(0x7ff708074d00 msgr2=0x7ff7080f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 -- 172.21.15.43:0/1402900656 shutdown_connections 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.280+0000 7ff70fed0640 1 -- 172.21.15.43:0/1402900656 wait complete. 2024-09-11T08:30:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70fed0640 1 Processor -- start 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70fed0640 1 -- start start 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70fed0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7080fb4a0 0x7ff708112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70fed0640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7081027e0 0x7ff708113430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70fed0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff708103180 0x7ff70810ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70fed0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70810d430 con 0x7ff7080fb4a0 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70fed0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70810d5a0 con 0x7ff708103180 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70fed0640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70810d760 con 0x7ff7081027e0 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70dc6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7080fb4a0 0x7ff708112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70d46b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7081027e0 0x7ff708113430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70dc6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7080fb4a0 0x7ff708112ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37328/0 (socket says 172.21.15.43:37328) 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70dc6c640 1 -- 172.21.15.43:0/2148343356 learned_addr learned my addr 172.21.15.43:0/2148343356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70e46d640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff708103180 0x7ff70810ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70dc6c640 1 -- 172.21.15.43:0/2148343356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7081027e0 msgr2=0x7ff708113430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70dc6c640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7081027e0 0x7ff708113430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70dc6c640 1 -- 172.21.15.43:0/2148343356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff708103180 msgr2=0x7ff70810ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70dc6c640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff708103180 0x7ff70810ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.281+0000 7ff70dc6c640 1 -- 172.21.15.43:0/2148343356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8005040 con 0x7ff7080fb4a0 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.282+0000 7ff70d46b640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7081027e0 0x7ff708113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.282+0000 7ff70dc6c640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7080fb4a0 0x7ff708112ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff70402d040 tx=0x7ff704035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.282+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/2148343356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704009850 con 0x7ff7080fb4a0 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.282+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff704005040 con 0x7ff7080fb4a0 2024-09-11T08:30:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.282+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/2148343356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7ff70402ab70 con 0x7ff7080fb4a0 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.282+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/2148343356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704033420 con 0x7ff7080fb4a0 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.282+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff708119d10 con 0x7ff7080fb4a0 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.283+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7080fe090 con 0x7ff7080fb4a0 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.284+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/2148343356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 90810+0+0 (secure 0 0 0) 0x7ff704030020 con 0x7ff7080fb4a0 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.284+0000 7ff6fe7fc640 1 --2- 172.21.15.43:0/2148343356 >> v2:172.21.15.43:6800/1424089977 conn(0x7ff6f406eeb0 0x7ff6f4071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.284+0000 7ff70d46b640 1 -- 172.21.15.43:0/2148343356 >> v2:172.21.15.43:6800/1424089977 conn(0x7ff6f406eeb0 msgr2=0x7ff6f4071360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1424089977 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.284+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/2148343356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5938+0+0 (secure 0 0 0) 0x7ff7040ae5d0 con 0x7ff7080fb4a0 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.284+0000 7ff70d46b640 1 --2- 172.21.15.43:0/2148343356 >> v2:172.21.15.43:6800/1424089977 conn(0x7ff6f406eeb0 0x7ff6f4071360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:30:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.293+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/2148343356 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff70407b710 con 0x7ff7080fb4a0 2024-09-11T08:30:38.471 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:38.299+0000 7fa3ae01bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:30:38.471 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:38.366+0000 7fa3ae01bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:30:38.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.485+0000 7ff70d46b640 1 -- 172.21.15.43:0/2148343356 >> v2:172.21.15.43:6800/1424089977 conn(0x7ff6f406eeb0 msgr2=0x7ff6f4071360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1424089977 2024-09-11T08:30:38.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.485+0000 7ff70d46b640 1 --2- 172.21.15.43:0/2148343356 >> v2:172.21.15.43:6800/1424089977 conn(0x7ff6f406eeb0 0x7ff6f4071360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T08:30:38.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.510+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff70804b720 con 0x7ff7080fb4a0 2024-09-11T08:30:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.513+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/2148343356 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+176668 (secure 0 0 0) 0x7ff70407b0b0 con 0x7ff7080fb4a0 2024-09-11T08:30:38.516 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:30:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 >> v2:172.21.15.43:6800/1424089977 conn(0x7ff6f406eeb0 msgr2=0x7ff6f4071360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:30:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 --2- 172.21.15.43:0/2148343356 >> v2:172.21.15.43:6800/1424089977 conn(0x7ff6f406eeb0 0x7ff6f4071360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7080fb4a0 msgr2=0x7ff708112ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7080fb4a0 0x7ff708112ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff70402d040 tx=0x7ff704035040 comp rx=0 tx=0).stop 2024-09-11T08:30:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 shutdown_connections 2024-09-11T08:30:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 --2- 172.21.15.43:0/2148343356 >> v2:172.21.15.43:6800/1424089977 conn(0x7ff6f406eeb0 0x7ff6f4071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7081027e0 0x7ff708113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff708103180 0x7ff70810ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 --2- 172.21.15.43:0/2148343356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7080fb4a0 0x7ff708112ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 >> 172.21.15.43:0/2148343356 conn(0x7ff708074d00 msgr2=0x7ff7081003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:38.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 shutdown_connections 2024-09-11T08:30:38.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:38.517+0000 7ff70fed0640 1 -- 172.21.15.43:0/2148343356 wait complete. 2024-09-11T08:30:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:38 smithi172 ceph-mon[32987]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:30:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:38 smithi172 ceph-mon[32987]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:30:38.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:38 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:38.252+0000 7ff3bd551ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:30:38.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:38 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:38.313+0000 7ff3bd551ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:30:38.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:38 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:38.478+0000 7ff3bd551ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:30:38.827 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:38 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:38.547+0000 7fa3ae01bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:30:38.828 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:38 smithi043 ceph-mon[40592]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:30:38.828 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:38 smithi043 ceph-mon[40592]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:30:38.828 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:38 smithi043 ceph-mon[29355]: from='mgr.14150 172.21.15.43:0/1409784313' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:30:38.828 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:38 smithi043 ceph-mon[29355]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:30:39.219 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1424089977}]},"active_addr":"172.21.15.43:6800/1424089977","active_change":"2024-09-11T08:25:39.290074+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1659353120}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2434949403}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":724821052}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2572823523}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":902711064}]}]} 2024-09-11T08:30:39.223 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T08:30:39.223 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T08:30:39.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:30:39.483 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2148343356' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:30:39.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:39 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:39.326+0000 7ff3bd551ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:30:39.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:39 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:39.391+0000 7ff3bd551ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:30:39.766 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:39 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:39.453+0000 7fa3ae01bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:30:39.766 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:39 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:39.530+0000 7fa3ae01bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:30:39.766 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2148343356' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:30:39.767 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2148343356' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:30:40.390 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.100+0000 7ff3bd551ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:30:40.390 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.211+0000 7ff3bd551ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:30:40.390 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.330+0000 7ff3bd551ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:30:40.390 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.390+0000 7ff3bd551ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:30:40.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:40.280+0000 7fa3ae01bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:30:40.641 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.515+0000 7ff3bd551ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:30:40.641 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.582+0000 7ff3bd551ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:30:40.641 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.640+0000 7ff3bd551ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:30:40.719 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:40.397+0000 7fa3ae01bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:30:40.719 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:40.525+0000 7fa3ae01bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:30:40.719 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:40.587+0000 7fa3ae01bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:30:40.903 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.775+0000 7ff3bd551ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:30:40.903 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.838+0000 7ff3bd551ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:30:40.903 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.903+0000 7ff3bd551ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:30:40.997 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:40.719+0000 7fa3ae01bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:30:40.997 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:40.791+0000 7fa3ae01bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:30:40.997 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:40.854+0000 7fa3ae01bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:30:40.997 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:40.996+0000 7fa3ae01bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:30:41.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:40.984+0000 7ff3bd551ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:30:41.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:41.041+0000 7ff3bd551ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:30:41.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:41.099+0000 7ff3bd551ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:30:41.248 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:41.068+0000 7fa3ae01bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:30:41.248 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:41.138+0000 7fa3ae01bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:30:41.248 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:41.225+0000 7fa3ae01bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 -- 172.21.15.43:0/2262230803 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f98101710 msgr2=0x7f6f98101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2262230803 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f98101710 0x7f6f98101ae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6f88004980 tx=0x7f6f88009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 -- 172.21.15.43:0/2262230803 shutdown_connections 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2262230803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f980fc0a0 0x7f6f981084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2262230803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f980fb6f0 0x7f6f980fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2262230803 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f98101710 0x7f6f98101ae0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 -- 172.21.15.43:0/2262230803 >> 172.21.15.43:0/2262230803 conn(0x7f6f98072420 msgr2=0x7f6f980f9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 -- 172.21.15.43:0/2262230803 shutdown_connections 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.244+0000 7f6f9da56640 1 -- 172.21.15.43:0/2262230803 wait complete. 2024-09-11T08:30:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f9da56640 1 Processor -- start 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f9da56640 1 -- start start 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f9da56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f980fb6f0 0x7f6f98111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f9da56640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f980fc0a0 0x7f6f981120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f9da56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f98101710 0x7f6f9810bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f9da56640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9810c3b0 con 0x7f6f980fb6f0 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f9da56640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9810c520 con 0x7f6f98101710 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f9da56640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9810c6e0 con 0x7f6f980fc0a0 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f980fb6f0 0x7f6f98111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f967fc640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f980fc0a0 0x7f6f981120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f967fc640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f980fc0a0 0x7f6f981120c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:47490/0 (socket says 172.21.15.43:47490) 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f967fc640 1 -- 172.21.15.43:0/2725442583 learned_addr learned my addr 172.21.15.43:0/2725442583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.245+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f980fb6f0 0x7f6f98111b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37344/0 (socket says 172.21.15.43:37344) 2024-09-11T08:30:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f977fe640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f98101710 0x7f6f9810bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f967fc640 1 -- 172.21.15.43:0/2725442583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f98101710 msgr2=0x7f6f9810bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f967fc640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f98101710 0x7f6f9810bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f967fc640 1 -- 172.21.15.43:0/2725442583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f980fb6f0 msgr2=0x7f6f98111b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f967fc640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f980fb6f0 0x7f6f98111b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f967fc640 1 -- 172.21.15.43:0/2725442583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f900098b0 con 0x7f6f980fc0a0 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f96ffd640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f980fb6f0 0x7f6f98111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f967fc640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f980fc0a0 0x7f6f981120c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c002b00 tx=0x7f6f8c004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f7f7fe640 1 -- 172.21.15.43:0/2725442583 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c007c30 con 0x7f6f980fc0a0 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f7f7fe640 1 -- 172.21.15.43:0/2725442583 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f6f8c00f040 con 0x7f6f980fc0a0 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f7f7fe640 1 -- 172.21.15.43:0/2725442583 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c009320 con 0x7f6f980fc0a0 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f88005040 con 0x7f6f980fc0a0 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.246+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f981189d0 con 0x7f6f980fc0a0 2024-09-11T08:30:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.247+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f980fce30 con 0x7f6f980fc0a0 2024-09-11T08:30:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.248+0000 7f6f7f7fe640 1 -- 172.21.15.43:0/2725442583 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 17) v1 ==== 90810+0+0 (secure 0 0 0) 0x7f6f8c009750 con 0x7f6f980fc0a0 2024-09-11T08:30:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.249+0000 7f6f7f7fe640 1 --2- 172.21.15.43:0/2725442583 >> v2:172.21.15.43:6800/1424089977 conn(0x7f6f8006ef60 0x7f6f80071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.249+0000 7f6f96ffd640 1 -- 172.21.15.43:0/2725442583 >> v2:172.21.15.43:6800/1424089977 conn(0x7f6f8006ef60 msgr2=0x7f6f80071410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1424089977 2024-09-11T08:30:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.249+0000 7f6f7f7fe640 1 -- 172.21.15.43:0/2725442583 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5938+0+0 (secure 0 0 0) 0x7f6f8c08e260 con 0x7f6f980fc0a0 2024-09-11T08:30:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.249+0000 7f6f96ffd640 1 --2- 172.21.15.43:0/2725442583 >> v2:172.21.15.43:6800/1424089977 conn(0x7f6f8006ef60 0x7f6f80071410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T08:30:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.253+0000 7f6f7f7fe640 1 -- 172.21.15.43:0/2725442583 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6f8c05b150 con 0x7f6f980fc0a0 2024-09-11T08:30:41.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.422+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f9810d220 con 0x7f6f980fc0a0 2024-09-11T08:30:41.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.423+0000 7f6f7f7fe640 1 -- 172.21.15.43:0/2725442583 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18890 (secure 0 0 0) 0x7f6f8c05aaf0 con 0x7f6f980fc0a0 2024-09-11T08:30:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:30:41.423 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"fsid":"327ee848-7017-11ef-bcea-c7b262605968","created":"2024-09-11T08:24:43.166587+0000","modified":"2024-09-11T08:30:14.055675+0000","last_up_change":"2024-09-11T08:29:48.817615+0000","last_in_change":"2024-09-11T08:29:32.826492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:27:53.800096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-11T08:30:01.538290+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-11T08:30:01.968623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:30:03.472929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:30:05.519250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:30:07.597076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"0752b9ca-c21a-45f5-9974-5942fd4e17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":3394646540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3394646540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3394646540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6803","nonce":3394646540}]},"public_addr":"172.21.15.43:6801/3394646540","cluster_addr":"172.21.15.43:6802/3394646540","heartbeat_back_addr":"172.21.15.43:6804/3394646540","heartbeat_front_addr":"172.21.15.43:6803/3394646540","state":["exists","up"]},{"osd":1,"uuid":"039df926-3c4b-48cd-9141-3a7684037575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":2076458881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2076458881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2076458881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6807","nonce":2076458881}]},"public_addr":"172.21.15.43:6805/2076458881","cluster_addr":"172.21.15.43:6806/2076458881","heartbeat_back_addr":"172.21.15.43:6808/2076458881","heartbeat_front_addr":"172.21.15.43:6807/2076458881","state":["exists","up"]},{"osd":2,"uuid":"d56b0955-f9bc-40f9-9fca-78ef5af1fd77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6809","nonce":319507648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":319507648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":319507648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":319507648}]},"public_addr":"172.21.15.43:6809/319507648","cluster_addr":"172.21.15.43:6810/319507648","heartbeat_back_addr":"172.21.15.43:6812/319507648","heartbeat_front_addr":"172.21.15.43:6811/319507648","state":["exists","up"]},{"osd":3,"uuid":"f653df6d-614c-4e93-b9dc-3f164fba74d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":87557626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":87557626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":87557626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":87557626}]},"public_addr":"172.21.15.43:6813/87557626","cluster_addr":"172.21.15.43:6814/87557626","heartbeat_back_addr":"172.21.15.43:6816/87557626","heartbeat_front_addr":"172.21.15.43:6815/87557626","state":["exists","up"]},{"osd":4,"uuid":"3083c1a6-6c94-47ed-bb13-86690fcebd11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":2140228578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6801","nonce":2140228578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6803","nonce":2140228578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2140228578}]},"public_addr":"172.21.15.172:6800/2140228578","cluster_addr":"172.21.15.172:6801/2140228578","heartbeat_back_addr":"172.21.15.172:6803/2140228578","heartbeat_front_addr":"172.21.15.172:6802/2140228578","state":["exists","up"]},{"osd":5,"uuid":"dab026a9-396f-4828-8d32-9c7e6142d1b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2848105378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6805","nonce":2848105378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6807","nonce":2848105378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2848105378}]},"public_addr":"172.21.15.172:6804/2848105378","cluster_addr":"172.21.15.172:6805/2848105378","heartbeat_back_addr":"172.21.15.172:6807/2848105378","heartbeat_front_addr":"172.21.15.172:6806/2848105378","state":["exists","up"]},{"osd":6,"uuid":"06710545-d1ff-4929-8dbe-97868b7dea5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2701192652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6809","nonce":2701192652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6811","nonce":2701192652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2701192652}]},"public_addr":"172.21.15.172:6808/2701192652","cluster_addr":"172.21.15.172:6809/2701192652","heartbeat_back_addr":"172.21.15.172:6811/2701192652","heartbeat_front_addr":"172.21.15.172:6810/2701192652","state":["exists","up"]},{"osd":7,"uuid":"23916580-474a-495f-9470-08f76d36069f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1466581331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6813","nonce":1466581331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6815","nonce":1466581331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1466581331}]},"public_addr":"172.21.15.172:6812/1466581331","cluster_addr":"172.21.15.172:6813/1466581331","heartbeat_back_addr":"172.21.15.172:6815/1466581331","heartbeat_front_addr":"172.21.15.172:6814/1466581331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:01.839914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:25.733653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:49.695899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:14.885633+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:36.661956+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:59.899466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:29:23.577485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:29:47.969389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6800/225101565":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/1607164598":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/162481557":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/848421812":"2024-09-12T08:25:15.636368+0000","172.21.15.43:0/1657974672":"2024-09-12T08:25:15.636368+0000","172.21.15.43:0/2577594813":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/2596169613":"2024-09-12T08:25:15.636368+0000","172.21.15.43:6800/3257998337":"2024-09-12T08:25:15.636368+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:30:41.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 >> v2:172.21.15.43:6800/1424089977 conn(0x7f6f8006ef60 msgr2=0x7f6f80071410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:30:41.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2725442583 >> v2:172.21.15.43:6800/1424089977 conn(0x7f6f8006ef60 0x7f6f80071410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f980fc0a0 msgr2=0x7f6f981120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:41.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f980fc0a0 0x7f6f981120c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c002b00 tx=0x7f6f8c004600 comp rx=0 tx=0).stop 2024-09-11T08:30:41.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 shutdown_connections 2024-09-11T08:30:41.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2725442583 >> v2:172.21.15.43:6800/1424089977 conn(0x7f6f8006ef60 0x7f6f80071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f980fc0a0 0x7f6f981120c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f98101710 0x7f6f9810bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 --2- 172.21.15.43:0/2725442583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f980fb6f0 0x7f6f98111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:41.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 >> 172.21.15.43:0/2725442583 conn(0x7f6f98072420 msgr2=0x7f6f980ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:41.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 shutdown_connections 2024-09-11T08:30:41.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:41.425+0000 7f6f9da56640 1 -- 172.21.15.43:0/2725442583 wait complete. 2024-09-11T08:30:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2725442583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:30:41.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:41.288+0000 7fa3ae01bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:30:41.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:41 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:41.349+0000 7fa3ae01bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:30:41.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2725442583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:30:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2725442583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:30:42.224 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T08:30:42.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:30:42.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:41.852+0000 7ff3bd551ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:30:42.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: [11/Sep/2024:08:30:41] ENGINE Bus STARTING 2024-09-11T08:30:42.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: CherryPy Checker: 2024-09-11T08:30:42.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: The Application mounted at '' has an empty config. 2024-09-11T08:30:42.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: 2024-09-11T08:30:42.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: [11/Sep/2024:08:30:41] ENGINE Serving on http://:::9283 2024-09-11T08:30:42.231 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: [11/Sep/2024:08:30:41] ENGINE Bus STARTED 2024-09-11T08:30:42.392 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:42.138+0000 7fa3ae01bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:30:42.393 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:30:42] ENGINE Bus STARTING 2024-09-11T08:30:42.393 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: CherryPy Checker: 2024-09-11T08:30:42.393 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: The Application mounted at '' has an empty config. 2024-09-11T08:30:42.393 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:42.393 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:30:42] ENGINE Serving on http://:::9283 2024-09-11T08:30:42.393 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:30:42] ENGINE Bus STARTED 2024-09-11T08:30:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: Standby manager daemon x restarted 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: Standby manager daemon x started 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: Active manager daemon y restarted 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: Activating manager daemon y 2024-09-11T08:30:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: mgrmap e18: y(active, starting, since 0.00788251s), standbys: x 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:30:42.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: Manager daemon y is now available 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:30:42.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: Standby manager daemon x restarted 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: Standby manager daemon x started 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: Active manager daemon y restarted 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: Activating manager daemon y 2024-09-11T08:30:42.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: mgrmap e18: y(active, starting, since 0.00788251s), standbys: x 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:30:42.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: Manager daemon y is now available 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:30:42.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:30:42.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: Standby manager daemon x restarted 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: Standby manager daemon x started 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.? 172.21.15.172:0/4040967129' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: Active manager daemon y restarted 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: Activating manager daemon y 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: mgrmap e18: y(active, starting, since 0.00788251s), standbys: x 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:30:42.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:30:42.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: Manager daemon y is now available 2024-09-11T08:30:42.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:30:42.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:42.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:30:42.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:30:42.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:30:42.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:30:42.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:30:44.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 -- 172.21.15.43:0/3744699050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 msgr2=0x7f61640fdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 --2- 172.21.15.43:0/3744699050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 0x7f61640fdba0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6158004950 tx=0x7f6158009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 -- 172.21.15.43:0/3744699050 shutdown_connections 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 --2- 172.21.15.43:0/3744699050 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61640fe170 0x7f6164106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 --2- 172.21.15.43:0/3744699050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6164106d80 0x7f6164109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 --2- 172.21.15.43:0/3744699050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 0x7f61640fdba0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 -- 172.21.15.43:0/3744699050 >> 172.21.15.43:0/3744699050 conn(0x7f6164072420 msgr2=0x7f61640f9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 -- 172.21.15.43:0/3744699050 shutdown_connections 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.293+0000 7f616bbc7640 1 -- 172.21.15.43:0/3744699050 wait complete. 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616bbc7640 1 Processor -- start 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616bbc7640 1 -- start start 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616bbc7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 0x7f61640fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616bbc7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61640fe170 0x7f61640fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616bbc7640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6164106d80 0x7f61640fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616bbc7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616411b2e0 con 0x7f61640fd7d0 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616bbc7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616411b450 con 0x7f61640fe170 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616bbc7640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616411b610 con 0x7f6164106d80 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f6169963640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 0x7f61640fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616a164640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6164106d80 0x7f61640fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616a164640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6164106d80 0x7f61640fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:42454/0 (socket says 172.21.15.43:42454) 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f6169162640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61640fe170 0x7f61640fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f616a164640 1 -- 172.21.15.43:0/2429708054 learned_addr learned my addr 172.21.15.43:0/2429708054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.294+0000 7f6169963640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 0x7f61640fb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57440/0 (socket says 172.21.15.43:57440) 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f616a164640 1 -- 172.21.15.43:0/2429708054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61640fe170 msgr2=0x7f61640fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f616a164640 1 --2- 172.21.15.43:0/2429708054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61640fe170 0x7f61640fbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f616a164640 1 -- 172.21.15.43:0/2429708054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 msgr2=0x7f61640fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f616a164640 1 --2- 172.21.15.43:0/2429708054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 0x7f61640fb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f616a164640 1 -- 172.21.15.43:0/2429708054 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160005040 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f6169963640 1 --2- 172.21.15.43:0/2429708054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 0x7f61640fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f616a164640 1 --2- 172.21.15.43:0/2429708054 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6164106d80 0x7f61640fc110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f615c00db50 tx=0x7f615c004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f61567fc640 1 -- 172.21.15.43:0/2429708054 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c017070 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f61567fc640 1 -- 172.21.15.43:0/2429708054 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f615c00f040 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f61567fc640 1 -- 172.21.15.43:0/2429708054 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c003780 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6158005040 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.295+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616411bbe0 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.296+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61640666e0 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.296+0000 7f61567fc640 1 -- 172.21.15.43:0/2429708054 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 19) v1 ==== 90554+0+0 (secure 0 0 0) 0x7f615c003bb0 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.297+0000 7f61567fc640 1 --2- 172.21.15.43:0/2429708054 >> v2:172.21.15.43:6800/588375333 conn(0x7f615006ec80 0x7f6150071130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.297+0000 7f6169963640 1 --2- 172.21.15.43:0/2429708054 >> v2:172.21.15.43:6800/588375333 conn(0x7f615006ec80 0x7f6150071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.297+0000 7f61567fc640 1 -- 172.21.15.43:0/2429708054 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f615c08df10 con 0x7f6164106d80 2024-09-11T08:30:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.298+0000 7f6169963640 1 --2- 172.21.15.43:0/2429708054 >> v2:172.21.15.43:6800/588375333 conn(0x7f615006ec80 0x7f6150071130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6158004920 tx=0x7f6158035040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.302+0000 7f61567fc640 1 -- 172.21.15.43:0/2429708054 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f615c0573a0 con 0x7f6164106d80 2024-09-11T08:30:44.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-mon[40592]: mgrmap e19: y(active, since 1.00995s), standbys: x 2024-09-11T08:30:44.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-mon[40592]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:44.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:44.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-mon[29355]: mgrmap e19: y(active, since 1.00995s), standbys: x 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-mon[29355]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:44.129+0000 7fa350d9d640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:30:44] ENGINE Error in HTTPServer.serve 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: Traceback (most recent call last): 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._connections.run(self.expiration_interval) 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._run(expiration_interval) 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:30:44.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: s = self.context.wrap_socket( 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: return self.sslsocket_class._create( 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self.do_handshake() 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: self._sslobj.do_handshake() 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:30:44.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:30:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:30:44.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.468+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f616410bc20 con 0x7f6164106d80 2024-09-11T08:30:44.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.469+0000 7f61567fc640 1 -- 172.21.15.43:0/2429708054 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+19220 (secure 0 0 0) 0x7f615c05abb0 con 0x7f6164106d80 2024-09-11T08:30:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:30:44.470 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"fsid":"327ee848-7017-11ef-bcea-c7b262605968","created":"2024-09-11T08:24:43.166587+0000","modified":"2024-09-11T08:30:42.140704+0000","last_up_change":"2024-09-11T08:29:48.817615+0000","last_in_change":"2024-09-11T08:29:32.826492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:27:53.800096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-11T08:30:01.538290+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-11T08:30:01.968623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:30:03.472929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:30:05.519250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:30:07.597076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"0752b9ca-c21a-45f5-9974-5942fd4e17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":3394646540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3394646540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3394646540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6803","nonce":3394646540}]},"public_addr":"172.21.15.43:6801/3394646540","cluster_addr":"172.21.15.43:6802/3394646540","heartbeat_back_addr":"172.21.15.43:6804/3394646540","heartbeat_front_addr":"172.21.15.43:6803/3394646540","state":["exists","up"]},{"osd":1,"uuid":"039df926-3c4b-48cd-9141-3a7684037575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":2076458881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2076458881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2076458881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6807","nonce":2076458881}]},"public_addr":"172.21.15.43:6805/2076458881","cluster_addr":"172.21.15.43:6806/2076458881","heartbeat_back_addr":"172.21.15.43:6808/2076458881","heartbeat_front_addr":"172.21.15.43:6807/2076458881","state":["exists","up"]},{"osd":2,"uuid":"d56b0955-f9bc-40f9-9fca-78ef5af1fd77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6809","nonce":319507648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":319507648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":319507648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":319507648}]},"public_addr":"172.21.15.43:6809/319507648","cluster_addr":"172.21.15.43:6810/319507648","heartbeat_back_addr":"172.21.15.43:6812/319507648","heartbeat_front_addr":"172.21.15.43:6811/319507648","state":["exists","up"]},{"osd":3,"uuid":"f653df6d-614c-4e93-b9dc-3f164fba74d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":87557626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":87557626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":87557626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":87557626}]},"public_addr":"172.21.15.43:6813/87557626","cluster_addr":"172.21.15.43:6814/87557626","heartbeat_back_addr":"172.21.15.43:6816/87557626","heartbeat_front_addr":"172.21.15.43:6815/87557626","state":["exists","up"]},{"osd":4,"uuid":"3083c1a6-6c94-47ed-bb13-86690fcebd11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":2140228578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6801","nonce":2140228578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6803","nonce":2140228578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2140228578}]},"public_addr":"172.21.15.172:6800/2140228578","cluster_addr":"172.21.15.172:6801/2140228578","heartbeat_back_addr":"172.21.15.172:6803/2140228578","heartbeat_front_addr":"172.21.15.172:6802/2140228578","state":["exists","up"]},{"osd":5,"uuid":"dab026a9-396f-4828-8d32-9c7e6142d1b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2848105378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6805","nonce":2848105378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6807","nonce":2848105378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2848105378}]},"public_addr":"172.21.15.172:6804/2848105378","cluster_addr":"172.21.15.172:6805/2848105378","heartbeat_back_addr":"172.21.15.172:6807/2848105378","heartbeat_front_addr":"172.21.15.172:6806/2848105378","state":["exists","up"]},{"osd":6,"uuid":"06710545-d1ff-4929-8dbe-97868b7dea5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2701192652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6809","nonce":2701192652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6811","nonce":2701192652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2701192652}]},"public_addr":"172.21.15.172:6808/2701192652","cluster_addr":"172.21.15.172:6809/2701192652","heartbeat_back_addr":"172.21.15.172:6811/2701192652","heartbeat_front_addr":"172.21.15.172:6810/2701192652","state":["exists","up"]},{"osd":7,"uuid":"23916580-474a-495f-9470-08f76d36069f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1466581331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6813","nonce":1466581331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6815","nonce":1466581331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1466581331}]},"public_addr":"172.21.15.172:6812/1466581331","cluster_addr":"172.21.15.172:6813/1466581331","heartbeat_back_addr":"172.21.15.172:6815/1466581331","heartbeat_front_addr":"172.21.15.172:6814/1466581331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:01.839914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:25.733653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:27:49.695899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:14.885633+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:36.661956+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:28:59.899466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:29:23.577485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:29:47.969389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2572823523":"2024-09-12T08:30:42.140652+0000","172.21.15.43:0/902711064":"2024-09-12T08:30:42.140652+0000","172.21.15.43:0/1657974672":"2024-09-12T08:25:15.636368+0000","172.21.15.43:0/162481557":"2024-09-12T08:25:39.289931+0000","172.21.15.43:6800/3257998337":"2024-09-12T08:25:15.636368+0000","172.21.15.43:0/848421812":"2024-09-12T08:25:15.636368+0000","172.21.15.43:0/2434949403":"2024-09-12T08:30:42.140652+0000","172.21.15.43:0/1607164598":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/2577594813":"2024-09-12T08:25:39.289931+0000","172.21.15.43:6800/225101565":"2024-09-12T08:25:39.289931+0000","172.21.15.43:0/2596169613":"2024-09-12T08:25:15.636368+0000","172.21.15.43:0/1659353120":"2024-09-12T08:30:42.140652+0000","172.21.15.43:0/724821052":"2024-09-12T08:30:42.140652+0000","172.21.15.43:6800/1424089977":"2024-09-12T08:30:42.140652+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:30:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 >> v2:172.21.15.43:6800/588375333 conn(0x7f615006ec80 msgr2=0x7f6150071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 --2- 172.21.15.43:0/2429708054 >> v2:172.21.15.43:6800/588375333 conn(0x7f615006ec80 0x7f6150071130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6158004920 tx=0x7f6158035040 comp rx=0 tx=0).stop 2024-09-11T08:30:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6164106d80 msgr2=0x7f61640fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 --2- 172.21.15.43:0/2429708054 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6164106d80 0x7f61640fc110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f615c00db50 tx=0x7f615c004e40 comp rx=0 tx=0).stop 2024-09-11T08:30:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 shutdown_connections 2024-09-11T08:30:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 --2- 172.21.15.43:0/2429708054 >> v2:172.21.15.43:6800/588375333 conn(0x7f615006ec80 0x7f6150071130 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 --2- 172.21.15.43:0/2429708054 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6164106d80 0x7f61640fc110 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 --2- 172.21.15.43:0/2429708054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61640fe170 0x7f61640fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 --2- 172.21.15.43:0/2429708054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61640fd7d0 0x7f61640fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:44.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 >> 172.21.15.43:0/2429708054 conn(0x7f6164072420 msgr2=0x7f61640fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:44.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 shutdown_connections 2024-09-11T08:30:44.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:44.471+0000 7f616bbc7640 1 -- 172.21.15.43:0/2429708054 wait complete. 2024-09-11T08:30:44.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:44 smithi172 ceph-mon[32987]: mgrmap e19: y(active, since 1.00995s), standbys: x 2024-09-11T08:30:44.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:44 smithi172 ceph-mon[32987]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:44.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:44 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:45.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T08:30:45.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T08:30:45.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T08:30:45.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T08:30:45.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T08:30:45.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T08:30:45.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T08:30:45.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T08:30:45.390 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-mon[32987]: [11/Sep/2024:08:30:44] ENGINE Bus STARTING 2024-09-11T08:30:45.390 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-mon[32987]: [11/Sep/2024:08:30:44] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-mon[32987]: [11/Sep/2024:08:30:44] ENGINE Bus STARTED 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-mon[32987]: [11/Sep/2024:08:30:44] ENGINE Error in HTTPServer.serve 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: Traceback (most recent call last): 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: self._run(expiration_interval) 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:30:45.391 INFO:journalctl@ceph.mon.b.smithi172.stdout: s = self.context.wrap_socket( 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout: return self.sslsocket_class._create( 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout: self.do_handshake() 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout: self._sslobj.do_handshake() 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-mon[32987]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2429708054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:30:45.392 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-mon[32987]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:30:45.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[40592]: [11/Sep/2024:08:30:44] ENGINE Bus STARTING 2024-09-11T08:30:45.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[40592]: [11/Sep/2024:08:30:44] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T08:30:45.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[40592]: [11/Sep/2024:08:30:44] ENGINE Bus STARTED 2024-09-11T08:30:45.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[40592]: [11/Sep/2024:08:30:44] ENGINE Error in HTTPServer.serve 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: Traceback (most recent call last): 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._run(expiration_interval) 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: s = self.context.wrap_socket( 2024-09-11T08:30:45.395 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout: return self.sslsocket_class._create( 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout: self.do_handshake() 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._sslobj.do_handshake() 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[40592]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2429708054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[40592]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[29355]: [11/Sep/2024:08:30:44] ENGINE Bus STARTING 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[29355]: [11/Sep/2024:08:30:44] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[29355]: [11/Sep/2024:08:30:44] ENGINE Bus STARTED 2024-09-11T08:30:45.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[29355]: [11/Sep/2024:08:30:44] ENGINE Error in HTTPServer.serve 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-11T08:30:45.397 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:30:45.398 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-11T08:30:45.398 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:30:45.398 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-11T08:30:45.398 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:30:45.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[29355]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:45.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2429708054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:30:45.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:45 smithi043 ceph-mon[29355]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:30:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:30:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:30:45.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:30:45.780 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:45.780 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.750 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:47.786 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:47 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi172 to 3760M 2024-09-11T08:30:47.786 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:47 smithi043 ceph-mon[40592]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:47.786 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:47 smithi043 ceph-mon[40592]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:30:47.787 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:47 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi172 to 3760M 2024-09-11T08:30:47.787 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:47 smithi043 ceph-mon[29355]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:47.787 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:47 smithi043 ceph-mon[29355]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:30:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:47 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi172 to 3760M 2024-09-11T08:30:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:47 smithi172 ceph-mon[32987]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:47 smithi172 ceph-mon[32987]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:30:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:30:49.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:49 smithi043 ceph-mon[40592]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:49.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:49 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:49.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:49 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:49.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:49 smithi043 ceph-mon[29355]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:49.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:49 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:49.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:49 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:49.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 -- 172.21.15.43:0/392304702 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa478102810 msgr2=0x7fa478102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:49.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 --2- 172.21.15.43:0/392304702 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa478102810 0x7fa478102be0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa46c004890 tx=0x7fa46c009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:49.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 -- 172.21.15.43:0/392304702 shutdown_connections 2024-09-11T08:30:49.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 --2- 172.21.15.43:0/392304702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4780fb440 0x7fa478109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:49.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 --2- 172.21.15.43:0/392304702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4781031b0 0x7fa4780fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 --2- 172.21.15.43:0/392304702 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa478102810 0x7fa478102be0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 -- 172.21.15.43:0/392304702 >> 172.21.15.43:0/392304702 conn(0x7fa478074c80 msgr2=0x7fa4780f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 -- 172.21.15.43:0/392304702 shutdown_connections 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.772+0000 7fa47fc8a640 1 -- 172.21.15.43:0/392304702 wait complete. 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47fc8a640 1 Processor -- start 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47fc8a640 1 -- start start 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47fc8a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4780fb440 0x7fa4780fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47fc8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa478102810 0x7fa4780fc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47fc8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4781031b0 0x7fa4780fc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47fc8a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47811b320 con 0x7fa4781031b0 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47fc8a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47811b490 con 0x7fa478102810 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47fc8a640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47811b650 con 0x7fa4780fb440 2024-09-11T08:30:49.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47da26640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4780fb440 0x7fa4780fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47e227640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4781031b0 0x7fa4780fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47da26640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4780fb440 0x7fa4780fbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:42480/0 (socket says 172.21.15.43:42480) 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47da26640 1 -- 172.21.15.43:0/3047830487 learned_addr learned my addr 172.21.15.43:0/3047830487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47e227640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4781031b0 0x7fa4780fc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57460/0 (socket says 172.21.15.43:57460) 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47d225640 1 --2- 172.21.15.43:0/3047830487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa478102810 0x7fa4780fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47da26640 1 -- 172.21.15.43:0/3047830487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa478102810 msgr2=0x7fa4780fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47da26640 1 --2- 172.21.15.43:0/3047830487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa478102810 0x7fa4780fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47da26640 1 -- 172.21.15.43:0/3047830487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4781031b0 msgr2=0x7fa4780fc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47da26640 1 --2- 172.21.15.43:0/3047830487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4781031b0 0x7fa4780fc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47da26640 1 -- 172.21.15.43:0/3047830487 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4700098b0 con 0x7fa4780fb440 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.773+0000 7fa47e227640 1 --2- 172.21.15.43:0/3047830487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4781031b0 0x7fa4780fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.774+0000 7fa47da26640 1 --2- 172.21.15.43:0/3047830487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4780fb440 0x7fa4780fbef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0048f0 tx=0x7fa46c009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.774+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0090d0 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.774+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fa46c009230 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.774+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c005040 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.774+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c033690 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.774+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47811bce0 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.775+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa478101e20 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.775+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fa46c030020 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.776+0000 7fa46a7fc640 1 --2- 172.21.15.43:0/3047830487 >> v2:172.21.15.43:6800/588375333 conn(0x7fa46406ef80 0x7fa464071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.776+0000 7fa47d225640 1 --2- 172.21.15.43:0/3047830487 >> v2:172.21.15.43:6800/588375333 conn(0x7fa46406ef80 0x7fa464071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.776+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7fa46c0ae5f0 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.776+0000 7fa46a7fc640 1 --2- 172.21.15.43:0/3047830487 >> v2:172.21.15.172:6812/1466581331 conn(0x7fa4640788b0 0x7fa46407ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.776+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 --> v2:172.21.15.172:6812/1466581331 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa46407b370 con 0x7fa4640788b0 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.776+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa46c0b0c80 con 0x7fa4780fb440 2024-09-11T08:30:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.776+0000 7fa47d225640 1 --2- 172.21.15.43:0/3047830487 >> v2:172.21.15.43:6800/588375333 conn(0x7fa46406ef80 0x7fa464071430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa4780fd620 tx=0x7fa47400b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.776+0000 7fa47e227640 1 --2- 172.21.15.43:0/3047830487 >> v2:172.21.15.172:6812/1466581331 conn(0x7fa4640788b0 0x7fa46407ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.777+0000 7fa47e227640 1 --2- 172.21.15.43:0/3047830487 >> v2:172.21.15.172:6812/1466581331 conn(0x7fa4640788b0 0x7fa46407ace0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.778+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 <== osd.7 v2:172.21.15.172:6812/1466581331 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa46407b370 con 0x7fa4640788b0 2024-09-11T08:30:49.787 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:49 smithi172 ceph-mon[32987]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:30:49.787 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:49 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:49.787 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:49 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:49.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.790+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 --> v2:172.21.15.172:6812/1466581331 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa4780fe030 con 0x7fa4640788b0 2024-09-11T08:30:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa46a7fc640 1 -- 172.21.15.43:0/3047830487 <== osd.7 v2:172.21.15.172:6812/1466581331 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa4780fe030 con 0x7fa4640788b0 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 >> v2:172.21.15.172:6812/1466581331 conn(0x7fa4640788b0 msgr2=0x7fa46407ace0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa47fc8a640 1 --2- 172.21.15.43:0/3047830487 >> v2:172.21.15.172:6812/1466581331 conn(0x7fa4640788b0 0x7fa46407ace0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 >> v2:172.21.15.43:6800/588375333 conn(0x7fa46406ef80 msgr2=0x7fa464071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa47fc8a640 1 --2- 172.21.15.43:0/3047830487 >> v2:172.21.15.43:6800/588375333 conn(0x7fa46406ef80 0x7fa464071430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa4780fd620 tx=0x7fa47400b040 comp rx=0 tx=0).stop 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4780fb440 msgr2=0x7fa4780fbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa47fc8a640 1 --2- 172.21.15.43:0/3047830487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4780fb440 0x7fa4780fbef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0048f0 tx=0x7fa46c009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa47e227640 1 -- 172.21.15.43:0/3047830487 reap_dead start 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.791+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 shutdown_connections 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.792+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 >> 172.21.15.43:0/3047830487 conn(0x7fa478074c80 msgr2=0x7fa478100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.792+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 shutdown_connections 2024-09-11T08:30:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:49.792+0000 7fa47fc8a640 1 -- 172.21.15.43:0/3047830487 wait complete. 2024-09-11T08:30:50.835 INFO:teuthology.orchestra.run.smithi043.stdout:210453397518 2024-09-11T08:30:50.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T08:30:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:51 smithi043 ceph-mon[40592]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:30:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:51 smithi043 ceph-mon[29355]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:30:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:51 smithi172 ceph-mon[32987]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:30:53.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:30:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:30:54.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:54 smithi043 ceph-mon[40592]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:30:54.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:54 smithi043 ceph-mon[40592]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:30:54.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:54 smithi043 ceph-mon[40592]: Deploying daemon alertmanager.a on smithi043 2024-09-11T08:30:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:54 smithi043 ceph-mon[29355]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:30:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:54 smithi043 ceph-mon[29355]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:30:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:54 smithi043 ceph-mon[29355]: Deploying daemon alertmanager.a on smithi043 2024-09-11T08:30:54.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:54 smithi172 ceph-mon[32987]: Updating smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:30:54.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:54 smithi172 ceph-mon[32987]: Updating smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:30:54.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:54 smithi172 ceph-mon[32987]: Deploying daemon alertmanager.a on smithi043 2024-09-11T08:30:55.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:55 smithi043 ceph-mon[40592]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:30:55.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:55 smithi043 ceph-mon[29355]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:30:55.401 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:55 smithi172 ceph-mon[32987]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 -- 172.21.15.43:0/218301985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46800fc0a0 msgr2=0x7f46801084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 --2- 172.21.15.43:0/218301985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46800fc0a0 0x7f46801084c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4678009d00 tx=0x7f4678004d10 comp rx=0 tx=0).stop 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 -- 172.21.15.43:0/218301985 shutdown_connections 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 --2- 172.21.15.43:0/218301985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46800fc0a0 0x7f46801084c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 --2- 172.21.15.43:0/218301985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46800fb6f0 0x7f46800fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 --2- 172.21.15.43:0/218301985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4680101710 0x7f4680101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 -- 172.21.15.43:0/218301985 >> 172.21.15.43:0/218301985 conn(0x7f4680071390 msgr2=0x7f46800f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 -- 172.21.15.43:0/218301985 shutdown_connections 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 -- 172.21.15.43:0/218301985 wait complete. 2024-09-11T08:30:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 Processor -- start 2024-09-11T08:30:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.568+0000 7f4687178640 1 -- start start 2024-09-11T08:30:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4687178640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46800fb6f0 0x7f4680112e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4687178640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46800fc0a0 0x7f4680113380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4687178640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4680101710 0x7f4680065170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4687178640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800657c0 con 0x7f4680101710 2024-09-11T08:30:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4687178640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4680065930 con 0x7f46800fc0a0 2024-09-11T08:30:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4687178640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4680065af0 con 0x7f46800fb6f0 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4684f14640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46800fb6f0 0x7f4680112e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4685715640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4680101710 0x7f4680065170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4684f14640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46800fb6f0 0x7f4680112e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49566/0 (socket says 172.21.15.43:49566) 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4684f14640 1 -- 172.21.15.43:0/2481819126 learned_addr learned my addr 172.21.15.43:0/2481819126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f467ffff640 1 --2- 172.21.15.43:0/2481819126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46800fc0a0 0x7f4680113380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4685715640 1 -- 172.21.15.43:0/2481819126 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46800fb6f0 msgr2=0x7f4680112e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4685715640 1 --2- 172.21.15.43:0/2481819126 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46800fb6f0 0x7f4680112e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4685715640 1 -- 172.21.15.43:0/2481819126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46800fc0a0 msgr2=0x7f4680113380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4685715640 1 --2- 172.21.15.43:0/2481819126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46800fc0a0 0x7f4680113380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:55.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4685715640 1 -- 172.21.15.43:0/2481819126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674005040 con 0x7f4680101710 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4684f14640 1 --2- 172.21.15.43:0/2481819126 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46800fb6f0 0x7f4680112e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.569+0000 7f4685715640 1 --2- 172.21.15.43:0/2481819126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4680101710 0x7f4680065170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f467802a910 tx=0x7f4678004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.570+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678038070 con 0x7f4680101710 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.570+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46780098b0 con 0x7f4680101710 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.570+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f4678033e90 con 0x7f4680101710 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.570+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467803c960 con 0x7f4680101710 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.570+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4680119c90 con 0x7f4680101710 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.570+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4680100be0 con 0x7f4680101710 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.571+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f4678030070 con 0x7f4680101710 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f467d7fa640 1 --2- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6800/588375333 conn(0x7f467006eed0 0x7f4670071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:55.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f46780aef00 con 0x7f4680101710 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f4684f14640 1 --2- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6800/588375333 conn(0x7f467006eed0 0x7f4670071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f467d7fa640 1 --2- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6813/87557626 conn(0x7f4670078780 0x7f467007abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 --> v2:172.21.15.43:6813/87557626 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f467007b240 con 0x7f4670078780 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46780af2d0 con 0x7f4680101710 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f467ffff640 1 --2- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6813/87557626 conn(0x7f4670078780 0x7f467007abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f4684f14640 1 --2- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6800/588375333 conn(0x7f467006eed0 0x7f4670071380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4674004f40 tx=0x7f467400b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.572+0000 7f467ffff640 1 --2- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6813/87557626 conn(0x7f4670078780 0x7f467007abb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.573+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 <== osd.3 v2:172.21.15.43:6813/87557626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f467007b240 con 0x7f4670078780 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.585+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 --> v2:172.21.15.43:6813/87557626 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f46800fce30 con 0x7f4670078780 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f467d7fa640 1 -- 172.21.15.43:0/2481819126 <== osd.3 v2:172.21.15.43:6813/87557626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f46800fce30 con 0x7f4670078780 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6813/87557626 conn(0x7f4670078780 msgr2=0x7f467007abb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 --2- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6813/87557626 conn(0x7f4670078780 0x7f467007abb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6800/588375333 conn(0x7f467006eed0 msgr2=0x7f4670071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 --2- 172.21.15.43:0/2481819126 >> v2:172.21.15.43:6800/588375333 conn(0x7f467006eed0 0x7f4670071380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4674004f40 tx=0x7f467400b040 comp rx=0 tx=0).stop 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4680101710 msgr2=0x7f4680065170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 --2- 172.21.15.43:0/2481819126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4680101710 0x7f4680065170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f467802a910 tx=0x7f4678004d10 comp rx=0 tx=0).stop 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4685715640 1 -- 172.21.15.43:0/2481819126 reap_dead start 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 shutdown_connections 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 >> 172.21.15.43:0/2481819126 conn(0x7f4680071390 msgr2=0x7f46800ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 shutdown_connections 2024-09-11T08:30:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.586+0000 7f4687178640 1 -- 172.21.15.43:0/2481819126 wait complete. 2024-09-11T08:30:55.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:30:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:30:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:30:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:30:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:30:56.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:56 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:56.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:56 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:56 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:30:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:57 smithi043 ceph-mon[40592]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:30:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:57 smithi043 ceph-mon[29355]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:30:57.229 INFO:teuthology.orchestra.run.smithi043.stdout:111669149729 2024-09-11T08:30:57.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T08:30:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:57 smithi172 ceph-mon[32987]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:30:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.019+0000 7f587ff20640 1 -- 172.21.15.43:0/2623651198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587806fe80 msgr2=0x7f58780702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.019+0000 7f587ff20640 1 --2- 172.21.15.43:0/2623651198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587806fe80 0x7f58780702f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5868004890 tx=0x7f5868009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 -- 172.21.15.43:0/2623651198 shutdown_connections 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 --2- 172.21.15.43:0/2623651198 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5878108f60 0x7f587810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 --2- 172.21.15.43:0/2623651198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587806fe80 0x7f58780702f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 --2- 172.21.15.43:0/2623651198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58780716f0 0x7f587806f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 -- 172.21.15.43:0/2623651198 >> 172.21.15.43:0/2623651198 conn(0x7f58780738c0 msgr2=0x7f5878073cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 -- 172.21.15.43:0/2623651198 shutdown_connections 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 -- 172.21.15.43:0/2623651198 wait complete. 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 Processor -- start 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 -- start start 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f587806fe80 0x7f5878114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58780716f0 0x7f5878115430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5878108f60 0x7f587810f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587810f720 con 0x7f58780716f0 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587810f890 con 0x7f5878108f60 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.020+0000 7f587ff20640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587810fa50 con 0x7f587806fe80 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f587806fe80 0x7f5878114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587d4bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58780716f0 0x7f5878115430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f587806fe80 0x7f5878114ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49546/0 (socket says 172.21.15.43:49546) 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587d4bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58780716f0 0x7f5878115430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51618/0 (socket says 172.21.15.43:51618) 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 -- 172.21.15.43:0/3275683412 learned_addr learned my addr 172.21.15.43:0/3275683412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587e4bd640 1 --2- 172.21.15.43:0/3275683412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5878108f60 0x7f587810f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 -- 172.21.15.43:0/3275683412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5878108f60 msgr2=0x7f587810f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 --2- 172.21.15.43:0/3275683412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5878108f60 0x7f587810f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 -- 172.21.15.43:0/3275683412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58780716f0 msgr2=0x7f5878115430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 --2- 172.21.15.43:0/3275683412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58780716f0 0x7f5878115430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 -- 172.21.15.43:0/3275683412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868005040 con 0x7f587806fe80 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587d4bb640 1 --2- 172.21.15.43:0/3275683412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58780716f0 0x7f5878115430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587e4bd640 1 --2- 172.21.15.43:0/3275683412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5878108f60 0x7f587810f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587dcbc640 1 --2- 172.21.15.43:0/3275683412 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f587806fe80 0x7f5878114ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f587400c890 tx=0x7f58740043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587400ce20 con 0x7f587806fe80 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587810fd30 con 0x7f587806fe80 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f587400acc0 con 0x7f587806fe80 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874012ca0 con 0x7f587806fe80 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.021+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587811bee0 con 0x7f587806fe80 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.022+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f587805dd90 con 0x7f587806fe80 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.023+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f587400a850 con 0x7f587806fe80 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.023+0000 7f586e7fc640 1 --2- 172.21.15.43:0/3275683412 >> v2:172.21.15.43:6800/588375333 conn(0x7f586406ef80 0x7f5864071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.023+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f587408e930 con 0x7f587806fe80 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.024+0000 7f587d4bb640 1 --2- 172.21.15.43:0/3275683412 >> v2:172.21.15.43:6800/588375333 conn(0x7f586406ef80 0x7f5864071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.024+0000 7f586e7fc640 1 --2- 172.21.15.43:0/3275683412 >> v2:172.21.15.172:6808/2701192652 conn(0x7f5864078830 0x7f586407ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.024+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 --> v2:172.21.15.172:6808/2701192652 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f586407b2f0 con 0x7f5864078830 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.024+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5874093240 con 0x7f587806fe80 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.024+0000 7f587d4bb640 1 --2- 172.21.15.43:0/3275683412 >> v2:172.21.15.43:6800/588375333 conn(0x7f586406ef80 0x7f5864071430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5868009040 tx=0x7f5868009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.024+0000 7f587e4bd640 1 --2- 172.21.15.43:0/3275683412 >> v2:172.21.15.172:6808/2701192652 conn(0x7f5864078830 0x7f586407ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.024+0000 7f587e4bd640 1 --2- 172.21.15.43:0/3275683412 >> v2:172.21.15.172:6808/2701192652 conn(0x7f5864078830 0x7f586407ac60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.025+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 <== osd.6 v2:172.21.15.172:6808/2701192652 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f586407b2f0 con 0x7f5864078830 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.038+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 --> v2:172.21.15.172:6808/2701192652 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5878070e50 con 0x7f5864078830 2024-09-11T08:30:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f586e7fc640 1 -- 172.21.15.43:0/3275683412 <== osd.6 v2:172.21.15.172:6808/2701192652 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5878070e50 con 0x7f5864078830 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 >> v2:172.21.15.172:6808/2701192652 conn(0x7f5864078830 msgr2=0x7f586407ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587ff20640 1 --2- 172.21.15.43:0/3275683412 >> v2:172.21.15.172:6808/2701192652 conn(0x7f5864078830 0x7f586407ac60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 >> v2:172.21.15.43:6800/588375333 conn(0x7f586406ef80 msgr2=0x7f5864071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587ff20640 1 --2- 172.21.15.43:0/3275683412 >> v2:172.21.15.43:6800/588375333 conn(0x7f586406ef80 0x7f5864071430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5868009040 tx=0x7f5868009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f587806fe80 msgr2=0x7f5878114ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587ff20640 1 --2- 172.21.15.43:0/3275683412 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f587806fe80 0x7f5878114ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f587400c890 tx=0x7f58740043c0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587e4bd640 1 -- 172.21.15.43:0/3275683412 reap_dead start 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 shutdown_connections 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.039+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 >> 172.21.15.43:0/3275683412 conn(0x7f58780738c0 msgr2=0x7f5878074b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.040+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 shutdown_connections 2024-09-11T08:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:55.040+0000 7f587ff20640 1 -- 172.21.15.43:0/3275683412 wait complete. 2024-09-11T08:30:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.906+0000 7f73baaa2640 1 -- 172.21.15.43:0/421823630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b4106d60 msgr2=0x7f73b4109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.906+0000 7f73baaa2640 1 --2- 172.21.15.43:0/421823630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b4106d60 0x7f73b4109140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f739c004950 tx=0x7f739c009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.906+0000 7f73baaa2640 1 -- 172.21.15.43:0/421823630 shutdown_connections 2024-09-11T08:30:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.906+0000 7f73baaa2640 1 --2- 172.21.15.43:0/421823630 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73b40fe150 0x7f73b4106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.906+0000 7f73baaa2640 1 --2- 172.21.15.43:0/421823630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b4106d60 0x7f73b4109140 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.906+0000 7f73baaa2640 1 --2- 172.21.15.43:0/421823630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b40fd7b0 0x7f73b40fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.906+0000 7f73baaa2640 1 -- 172.21.15.43:0/421823630 >> 172.21.15.43:0/421823630 conn(0x7f73b4071390 msgr2=0x7f73b40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.906+0000 7f73baaa2640 1 -- 172.21.15.43:0/421823630 shutdown_connections 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 -- 172.21.15.43:0/421823630 wait complete. 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 Processor -- start 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 -- start start 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b40fd7b0 0x7f73b40fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73b40fe150 0x7f73b40fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b4106d60 0x7f73b411b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40fc330 con 0x7f73b4106d60 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40fc4a0 con 0x7f73b40fd7b0 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.907+0000 7f73baaa2640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b411b6e0 con 0x7f73b40fe150 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73b40fe150 0x7f73b40fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b903f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b4106d60 0x7f73b411b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73b40fe150 0x7f73b40fbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49594/0 (socket says 172.21.15.43:49594) 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b3fff640 1 -- 172.21.15.43:0/2265983063 learned_addr learned my addr 172.21.15.43:0/2265983063 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b883e640 1 --2- 172.21.15.43:0/2265983063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b40fd7b0 0x7f73b40fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b903f640 1 -- 172.21.15.43:0/2265983063 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73b40fe150 msgr2=0x7f73b40fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b903f640 1 --2- 172.21.15.43:0/2265983063 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73b40fe150 0x7f73b40fbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b903f640 1 -- 172.21.15.43:0/2265983063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b40fd7b0 msgr2=0x7f73b40fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b903f640 1 --2- 172.21.15.43:0/2265983063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b40fd7b0 0x7f73b40fb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b903f640 1 -- 172.21.15.43:0/2265983063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a40098b0 con 0x7f73b4106d60 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b3fff640 1 --2- 172.21.15.43:0/2265983063 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73b40fe150 0x7f73b40fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b903f640 1 --2- 172.21.15.43:0/2265983063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b4106d60 0x7f73b411b150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f739c0047f0 tx=0x7f739c035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c004160 con 0x7f73b4106d60 2024-09-11T08:30:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c005040 con 0x7f73b4106d60 2024-09-11T08:30:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f739c02ab70 con 0x7f73b4106d60 2024-09-11T08:30:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c033690 con 0x7f73b4106d60 2024-09-11T08:30:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.908+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b411bcc0 con 0x7f73b4106d60 2024-09-11T08:30:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.909+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f73b405ddd0 con 0x7f73b4106d60 2024-09-11T08:30:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.910+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f739c030020 con 0x7f73b4106d60 2024-09-11T08:30:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.910+0000 7f73b17fa640 1 --2- 172.21.15.43:0/2265983063 >> v2:172.21.15.43:6800/588375333 conn(0x7f73a006eed0 0x7f73a0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.910+0000 7f73b883e640 1 --2- 172.21.15.43:0/2265983063 >> v2:172.21.15.43:6800/588375333 conn(0x7f73a006eed0 0x7f73a0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.910+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f739c0ae860 con 0x7f73b4106d60 2024-09-11T08:30:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.911+0000 7f73b17fa640 1 --2- 172.21.15.43:0/2265983063 >> v2:172.21.15.172:6804/2848105378 conn(0x7f73a0078800 0x7f73a007ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.911+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 --> v2:172.21.15.172:6804/2848105378 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73a007b2c0 con 0x7f73a0078800 2024-09-11T08:30:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.911+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f739c0b0dc0 con 0x7f73b4106d60 2024-09-11T08:30:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.911+0000 7f73b883e640 1 --2- 172.21.15.43:0/2265983063 >> v2:172.21.15.43:6800/588375333 conn(0x7f73a006eed0 0x7f73a0071380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009470 tx=0x7f73ac009ed0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.911+0000 7f73b3fff640 1 --2- 172.21.15.43:0/2265983063 >> v2:172.21.15.172:6804/2848105378 conn(0x7f73a0078800 0x7f73a007ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.911+0000 7f73b3fff640 1 --2- 172.21.15.43:0/2265983063 >> v2:172.21.15.172:6804/2848105378 conn(0x7f73a0078800 0x7f73a007ac30 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.912+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 <== osd.5 v2:172.21.15.172:6804/2848105378 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f73a007b2c0 con 0x7f73a0078800 2024-09-11T08:30:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 --> v2:172.21.15.172:6804/2848105378 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f73b4065970 con 0x7f73a0078800 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73b17fa640 1 -- 172.21.15.43:0/2265983063 <== osd.5 v2:172.21.15.172:6804/2848105378 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f73b4065970 con 0x7f73a0078800 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 >> v2:172.21.15.172:6804/2848105378 conn(0x7f73a0078800 msgr2=0x7f73a007ac30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73baaa2640 1 --2- 172.21.15.43:0/2265983063 >> v2:172.21.15.172:6804/2848105378 conn(0x7f73a0078800 0x7f73a007ac30 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 >> v2:172.21.15.43:6800/588375333 conn(0x7f73a006eed0 msgr2=0x7f73a0071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73baaa2640 1 --2- 172.21.15.43:0/2265983063 >> v2:172.21.15.43:6800/588375333 conn(0x7f73a006eed0 0x7f73a0071380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009470 tx=0x7f73ac009ed0 comp rx=0 tx=0).stop 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b4106d60 msgr2=0x7f73b411b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73baaa2640 1 --2- 172.21.15.43:0/2265983063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b4106d60 0x7f73b411b150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f739c0047f0 tx=0x7f739c035040 comp rx=0 tx=0).stop 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.925+0000 7f73b903f640 1 -- 172.21.15.43:0/2265983063 reap_dead start 2024-09-11T08:30:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.926+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 shutdown_connections 2024-09-11T08:30:57.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.926+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 >> 172.21.15.43:0/2265983063 conn(0x7f73b4071390 msgr2=0x7f73b4072690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:57.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.926+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 shutdown_connections 2024-09-11T08:30:57.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:57.926+0000 7f73baaa2640 1 -- 172.21.15.43:0/2265983063 wait complete. 2024-09-11T08:30:58.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:58 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:30:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:58 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:30:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:58 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:30:58.572 INFO:teuthology.orchestra.run.smithi043.stdout:184683593748 2024-09-11T08:30:58.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T08:30:59.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 -- 172.21.15.43:0/3091447146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 msgr2=0x7f87ac0fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 --2- 172.21.15.43:0/3091447146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 0x7f87ac0fcc50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87a0004980 tx=0x7f87a0009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 -- 172.21.15.43:0/3091447146 shutdown_connections 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 --2- 172.21.15.43:0/3091447146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 0x7f87ac0fcc50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 --2- 172.21.15.43:0/3091447146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87ac0fd190 0x7f87ac1095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 --2- 172.21.15.43:0/3091447146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ac102800 0x7f87ac102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 -- 172.21.15.43:0/3091447146 >> 172.21.15.43:0/3091447146 conn(0x7f87ac073430 msgr2=0x7f87ac0faac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 -- 172.21.15.43:0/3091447146 shutdown_connections 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.977+0000 7f87b137b640 1 -- 172.21.15.43:0/3091447146 wait complete. 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87b137b640 1 Processor -- start 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87b137b640 1 -- start start 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87b137b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 0x7f87ac112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87b137b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87ac0fd190 0x7f87ac113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87b137b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ac102800 0x7f87ac10cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87b137b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac10d490 con 0x7f87ac102800 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87b137b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac10d600 con 0x7f87ac0fd190 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87b137b640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac10d7c0 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87aaffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 0x7f87ac112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87ab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ac102800 0x7f87ac10cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87aaffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 0x7f87ac112cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49612/0 (socket says 172.21.15.43:49612) 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.978+0000 7f87aaffd640 1 -- 172.21.15.43:0/1674660662 learned_addr learned my addr 172.21.15.43:0/1674660662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87aa7fc640 1 --2- 172.21.15.43:0/1674660662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87ac0fd190 0x7f87ac113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87aaffd640 1 -- 172.21.15.43:0/1674660662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87ac0fd190 msgr2=0x7f87ac113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87aaffd640 1 --2- 172.21.15.43:0/1674660662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87ac0fd190 0x7f87ac113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87aaffd640 1 -- 172.21.15.43:0/1674660662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ac102800 msgr2=0x7f87ac10cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87aaffd640 1 --2- 172.21.15.43:0/1674660662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ac102800 0x7f87ac10cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87aaffd640 1 -- 172.21.15.43:0/1674660662 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0098b0 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87aa7fc640 1 --2- 172.21.15.43:0/1674660662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87ac0fd190 0x7f87ac113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87ab7fe640 1 --2- 172.21.15.43:0/1674660662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ac102800 0x7f87ac10cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87aaffd640 1 --2- 172.21.15.43:0/1674660662 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 0x7f87ac112cc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f879c001f00 tx=0x7f879c004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c00fea0 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f879c007d00 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a0005040 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c015470 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.979+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac119ae0 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.980+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87ac101d10 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.981+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f879c007890 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.982+0000 7f87937fe640 1 --2- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6800/588375333 conn(0x7f879406ef80 0x7f8794071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.982+0000 7f87aa7fc640 1 --2- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6800/588375333 conn(0x7f879406ef80 0x7f8794071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.982+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f879c08e8a0 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.982+0000 7f87937fe640 1 --2- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6805/2076458881 conn(0x7f87940788b0 0x7f879407ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.982+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 --> v2:172.21.15.43:6805/2076458881 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f879407b370 con 0x7f87940788b0 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.982+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f879c093240 con 0x7f87ac0fc7e0 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.982+0000 7f87ab7fe640 1 --2- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6805/2076458881 conn(0x7f87940788b0 0x7f879407ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.982+0000 7f87aa7fc640 1 --2- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6800/588375333 conn(0x7f879406ef80 0x7f8794071430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009f80 tx=0x7f87a0009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.983+0000 7f87ab7fe640 1 --2- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6805/2076458881 conn(0x7f87940788b0 0x7f879407ace0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.984+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 <== osd.1 v2:172.21.15.43:6805/2076458881 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f879407b370 con 0x7f87940788b0 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 --> v2:172.21.15.43:6805/2076458881 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f87ac0fdf20 con 0x7f87940788b0 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87937fe640 1 -- 172.21.15.43:0/1674660662 <== osd.1 v2:172.21.15.43:6805/2076458881 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f87ac0fdf20 con 0x7f87940788b0 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6805/2076458881 conn(0x7f87940788b0 msgr2=0x7f879407ace0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 --2- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6805/2076458881 conn(0x7f87940788b0 0x7f879407ace0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6800/588375333 conn(0x7f879406ef80 msgr2=0x7f8794071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 --2- 172.21.15.43:0/1674660662 >> v2:172.21.15.43:6800/588375333 conn(0x7f879406ef80 0x7f8794071430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009f80 tx=0x7f87a0009da0 comp rx=0 tx=0).stop 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 msgr2=0x7f87ac112cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 --2- 172.21.15.43:0/1674660662 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87ac0fc7e0 0x7f87ac112cc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f879c001f00 tx=0x7f879c004c50 comp rx=0 tx=0).stop 2024-09-11T08:30:59.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87ab7fe640 1 -- 172.21.15.43:0/1674660662 reap_dead start 2024-09-11T08:30:59.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 shutdown_connections 2024-09-11T08:30:59.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 >> 172.21.15.43:0/1674660662 conn(0x7f87ac073430 msgr2=0x7f87ac100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:30:59.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 shutdown_connections 2024-09-11T08:30:59.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:30:58.999+0000 7f87b137b640 1 -- 172.21.15.43:0/1674660662 wait complete. 2024-09-11T08:30:59.087 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:30:59 smithi043 ceph-mon[40592]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:30:59.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:30:59 smithi043 ceph-mon[29355]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:30:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:30:59 smithi172 ceph-mon[32987]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:31:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.051+0000 7f8d11271640 1 -- 172.21.15.43:0/3786813877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c0fe190 msgr2=0x7f8d0c106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.051+0000 7f8d11271640 1 --2- 172.21.15.43:0/3786813877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c0fe190 0x7f8d0c106860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d00004950 tx=0x7f8d00009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.051+0000 7f8d11271640 1 -- 172.21.15.43:0/3786813877 shutdown_connections 2024-09-11T08:31:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.051+0000 7f8d11271640 1 --2- 172.21.15.43:0/3786813877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c0fe190 0x7f8d0c106860 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.051+0000 7f8d11271640 1 --2- 172.21.15.43:0/3786813877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d0c106da0 0x7f8d0c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.051+0000 7f8d11271640 1 --2- 172.21.15.43:0/3786813877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d0c0fd7f0 0x7f8d0c0fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.051+0000 7f8d11271640 1 -- 172.21.15.43:0/3786813877 >> 172.21.15.43:0/3786813877 conn(0x7f8d0c071390 msgr2=0x7f8d0c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.052+0000 7f8d11271640 1 -- 172.21.15.43:0/3786813877 shutdown_connections 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.052+0000 7f8d11271640 1 -- 172.21.15.43:0/3786813877 wait complete. 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.052+0000 7f8d11271640 1 Processor -- start 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.052+0000 7f8d11271640 1 -- start start 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d11271640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d0c0fd7f0 0x7f8d0c113000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d11271640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d0c0fe190 0x7f8d0c113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d11271640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c106da0 0x7f8d0c10cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d11271640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c10d5d0 con 0x7f8d0c0fe190 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d11271640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c10d740 con 0x7f8d0c0fd7f0 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d11271640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c10d900 con 0x7f8d0c106da0 2024-09-11T08:31:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d0c0fe190 0x7f8d0c113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0b7fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c106da0 0x7f8d0c10cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d0c0fe190 0x7f8d0c113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51666/0 (socket says 172.21.15.43:51666) 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0b7fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c106da0 0x7f8d0c10cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49632/0 (socket says 172.21.15.43:49632) 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0a7fc640 1 -- 172.21.15.43:0/1290179483 learned_addr learned my addr 172.21.15.43:0/1290179483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0affd640 1 --2- 172.21.15.43:0/1290179483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d0c0fd7f0 0x7f8d0c113000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0b7fe640 1 -- 172.21.15.43:0/1290179483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d0c0fd7f0 msgr2=0x7f8d0c113000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0b7fe640 1 --2- 172.21.15.43:0/1290179483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d0c0fd7f0 0x7f8d0c113000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0b7fe640 1 -- 172.21.15.43:0/1290179483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d0c0fe190 msgr2=0x7f8d0c113540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0b7fe640 1 --2- 172.21.15.43:0/1290179483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d0c0fe190 0x7f8d0c113540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0b7fe640 1 -- 172.21.15.43:0/1290179483 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04005040 con 0x7f8d0c106da0 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0a7fc640 1 --2- 172.21.15.43:0/1290179483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d0c0fe190 0x7f8d0c113540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.053+0000 7f8d0b7fe640 1 --2- 172.21.15.43:0/1290179483 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c106da0 0x7f8d0c10cf80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8d040093d0 tx=0x7f8d04009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.054+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04007b20 con 0x7f8d0c106da0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.054+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d00005040 con 0x7f8d0c106da0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.054+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8d0400f040 con 0x7f8d0c106da0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.054+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04013a50 con 0x7f8d0c106da0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.054+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c119e50 con 0x7f8d0c106da0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.055+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d0c05ddd0 con 0x7f8d0c106da0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.055+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f8d0400e930 con 0x7f8d0c106da0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8cf37fe640 1 --2- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6800/588375333 conn(0x7f8cf406eed0 0x7f8cf4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f8d0408e480 con 0x7f8d0c106da0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8d0affd640 1 --2- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6800/588375333 conn(0x7f8cf406eed0 0x7f8cf4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8cf37fe640 1 --2- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6801/3394646540 conn(0x7f8cf4078780 0x7f8cf407abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 --> v2:172.21.15.43:6801/3394646540 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cf407b240 con 0x7f8cf4078780 2024-09-11T08:31:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d04090ac0 con 0x7f8d0c106da0 2024-09-11T08:31:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8d0a7fc640 1 --2- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6801/3394646540 conn(0x7f8cf4078780 0x7f8cf407abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8d0affd640 1 --2- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6800/588375333 conn(0x7f8cf406eed0 0x7f8cf4071380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc004550 tx=0x7f8cfc009040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.056+0000 7f8d0a7fc640 1 --2- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6801/3394646540 conn(0x7f8cf4078780 0x7f8cf407abb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.057+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 <== osd.0 v2:172.21.15.43:6801/3394646540 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8cf407b240 con 0x7f8cf4078780 2024-09-11T08:31:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 --> v2:172.21.15.43:6801/3394646540 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d0c065930 con 0x7f8cf4078780 2024-09-11T08:31:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8cf37fe640 1 -- 172.21.15.43:0/1290179483 <== osd.0 v2:172.21.15.43:6801/3394646540 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8d0c065930 con 0x7f8cf4078780 2024-09-11T08:31:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6801/3394646540 conn(0x7f8cf4078780 msgr2=0x7f8cf407abb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 --2- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6801/3394646540 conn(0x7f8cf4078780 0x7f8cf407abb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6800/588375333 conn(0x7f8cf406eed0 msgr2=0x7f8cf4071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 --2- 172.21.15.43:0/1290179483 >> v2:172.21.15.43:6800/588375333 conn(0x7f8cf406eed0 0x7f8cf4071380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc004550 tx=0x7f8cfc009040 comp rx=0 tx=0).stop 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c106da0 msgr2=0x7f8d0c10cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 --2- 172.21.15.43:0/1290179483 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d0c106da0 0x7f8d0c10cf80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8d040093d0 tx=0x7f8d04009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d0b7fe640 1 -- 172.21.15.43:0/1290179483 reap_dead start 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 shutdown_connections 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 >> 172.21.15.43:0/1290179483 conn(0x7f8d0c071390 msgr2=0x7f8d0c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 shutdown_connections 2024-09-11T08:31:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.070+0000 7f8d11271640 1 -- 172.21.15.43:0/1290179483 wait complete. 2024-09-11T08:31:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 -- 172.21.15.43:0/916463650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fd410 msgr2=0x7f27fc1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 --2- 172.21.15.43:0/916463650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fd410 0x7f27fc1095b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009d00 tx=0x7f27f4004d10 comp rx=0 tx=0).stop 2024-09-11T08:31:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 -- 172.21.15.43:0/916463650 shutdown_connections 2024-09-11T08:31:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 --2- 172.21.15.43:0/916463650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fd410 0x7f27fc1095b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 --2- 172.21.15.43:0/916463650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27fc0fca60 0x7f27fc0fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 --2- 172.21.15.43:0/916463650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc1028a0 0x7f27fc102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 -- 172.21.15.43:0/916463650 >> 172.21.15.43:0/916463650 conn(0x7f27fc0738c0 msgr2=0x7f27fc073cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 -- 172.21.15.43:0/916463650 shutdown_connections 2024-09-11T08:31:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.706+0000 7f2801818640 1 -- 172.21.15.43:0/916463650 wait complete. 2024-09-11T08:31:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f2801818640 1 Processor -- start 2024-09-11T08:31:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f2801818640 1 -- start start 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f2801818640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fca60 0x7f27fc06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f2801818640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc0fd410 0x7f27fc06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f2801818640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27fc1028a0 0x7f27fc070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f2801818640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc11b0b0 con 0x7f27fc1028a0 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f2801818640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc11b220 con 0x7f27fc0fd410 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f2801818640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc11b3e0 con 0x7f27fc0fca60 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27fc1028a0 0x7f27fc070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f27faffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fca60 0x7f27fc06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f27faffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fca60 0x7f27fc06fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49658/0 (socket says 172.21.15.43:49658) 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27fc1028a0 0x7f27fc070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51696/0 (socket says 172.21.15.43:51696) 2024-09-11T08:31:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f27faffd640 1 -- 172.21.15.43:0/2157478541 learned_addr learned my addr 172.21.15.43:0/2157478541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.707+0000 7f27fa7fc640 1 --2- 172.21.15.43:0/2157478541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc0fd410 0x7f27fc06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27faffd640 1 -- 172.21.15.43:0/2157478541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc0fd410 msgr2=0x7f27fc06ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27faffd640 1 --2- 172.21.15.43:0/2157478541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc0fd410 0x7f27fc06ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27faffd640 1 -- 172.21.15.43:0/2157478541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27fc1028a0 msgr2=0x7f27fc070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27faffd640 1 --2- 172.21.15.43:0/2157478541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27fc1028a0 0x7f27fc070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27faffd640 1 -- 172.21.15.43:0/2157478541 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f0005040 con 0x7f27fc0fca60 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27fb7fe640 1 --2- 172.21.15.43:0/2157478541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27fc1028a0 0x7f27fc070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27fa7fc640 1 --2- 172.21.15.43:0/2157478541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc0fd410 0x7f27fc06ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27faffd640 1 --2- 172.21.15.43:0/2157478541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fca60 0x7f27fc06fa10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00c890 tx=0x7f27ec0043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec00ce20 con 0x7f27fc0fca60 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f27ec00acc0 con 0x7f27fc0fca60 2024-09-11T08:31:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f40098b0 con 0x7f27fc0fca60 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec012ca0 con 0x7f27fc0fca60 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.708+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc11ba00 con 0x7f27fc0fca60 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.709+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27fc1016a0 con 0x7f27fc0fca60 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.710+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f27ec00a850 con 0x7f27fc0fca60 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.710+0000 7f27e37fe640 1 --2- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6800/588375333 conn(0x7f27e406ef80 0x7f27e4071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.710+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f27ec08e930 con 0x7f27fc0fca60 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.710+0000 7f27fa7fc640 1 --2- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6800/588375333 conn(0x7f27e406ef80 0x7f27e4071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.710+0000 7f27e37fe640 1 --2- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6809/319507648 conn(0x7f27e4078830 0x7f27e407ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.710+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 --> v2:172.21.15.43:6809/319507648 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27e407b2f0 con 0x7f27e4078830 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.710+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27ec093240 con 0x7f27fc0fca60 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.710+0000 7f27fb7fe640 1 --2- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6809/319507648 conn(0x7f27e4078830 0x7f27e407ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.711+0000 7f27fa7fc640 1 --2- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6800/588375333 conn(0x7f27e406ef80 0x7f27e4071430 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f27f0007fd0 tx=0x7f27f0009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.711+0000 7f27fb7fe640 1 --2- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6809/319507648 conn(0x7f27e4078830 0x7f27e407ac60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.711+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 <== osd.2 v2:172.21.15.43:6809/319507648 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f27e407b2f0 con 0x7f27e4078830 2024-09-11T08:31:00.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.724+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 --> v2:172.21.15.43:6809/319507648 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f27fc0fdfc0 con 0x7f27e4078830 2024-09-11T08:31:00.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.724+0000 7f27e37fe640 1 -- 172.21.15.43:0/2157478541 <== osd.2 v2:172.21.15.43:6809/319507648 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f27fc0fdfc0 con 0x7f27e4078830 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.724+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6809/319507648 conn(0x7f27e4078830 msgr2=0x7f27e407ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.724+0000 7f2801818640 1 --2- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6809/319507648 conn(0x7f27e4078830 0x7f27e407ac60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6800/588375333 conn(0x7f27e406ef80 msgr2=0x7f27e4071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f2801818640 1 --2- 172.21.15.43:0/2157478541 >> v2:172.21.15.43:6800/588375333 conn(0x7f27e406ef80 0x7f27e4071430 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f27f0007fd0 tx=0x7f27f0009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fca60 msgr2=0x7f27fc06fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f2801818640 1 --2- 172.21.15.43:0/2157478541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27fc0fca60 0x7f27fc06fa10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00c890 tx=0x7f27ec0043c0 comp rx=0 tx=0).stop 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f27fb7fe640 1 -- 172.21.15.43:0/2157478541 reap_dead start 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 shutdown_connections 2024-09-11T08:31:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 >> 172.21.15.43:0/2157478541 conn(0x7f27fc0738c0 msgr2=0x7f27fc0fb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:00.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 shutdown_connections 2024-09-11T08:31:00.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:00.725+0000 7f2801818640 1 -- 172.21.15.43:0/2157478541 wait complete. 2024-09-11T08:31:01.208 INFO:teuthology.orchestra.run.smithi043.stdout:34359738416 2024-09-11T08:31:01.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T08:31:01.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:01 smithi043 ceph-mon[40592]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:31:01.435 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:01 smithi043 ceph-mon[29355]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:31:01.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 -- 172.21.15.43:0/170387441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc106df0 msgr2=0x7f36cc1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 --2- 172.21.15.43:0/170387441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc106df0 0x7f36cc1091d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009cd0 tx=0x7f36c4004d10 comp rx=0 tx=0).stop 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 -- 172.21.15.43:0/170387441 shutdown_connections 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 --2- 172.21.15.43:0/170387441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc106df0 0x7f36cc1091d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 --2- 172.21.15.43:0/170387441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36cc0fe1e0 0x7f36cc1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 --2- 172.21.15.43:0/170387441 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36cc0fd840 0x7f36cc0fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 -- 172.21.15.43:0/170387441 >> 172.21.15.43:0/170387441 conn(0x7f36cc072580 msgr2=0x7f36cc0f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 -- 172.21.15.43:0/170387441 shutdown_connections 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.430+0000 7f36d3564640 1 -- 172.21.15.43:0/170387441 wait complete. 2024-09-11T08:31:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d3564640 1 Processor -- start 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d3564640 1 -- start start 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d3564640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36cc0fd840 0x7f36cc112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d3564640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc0fe1e0 0x7f36cc112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d3564640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36cc106df0 0x7f36cc10cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d3564640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc10d290 con 0x7f36cc0fe1e0 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d3564640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc10d400 con 0x7f36cc0fd840 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d3564640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc10d5c0 con 0x7f36cc106df0 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d0aff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc0fe1e0 0x7f36cc112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d1b01640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36cc106df0 0x7f36cc10cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d0aff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc0fe1e0 0x7f36cc112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51702/0 (socket says 172.21.15.43:51702) 2024-09-11T08:31:01.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d0aff640 1 -- 172.21.15.43:0/3171329995 learned_addr learned my addr 172.21.15.43:0/3171329995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d1300640 1 --2- 172.21.15.43:0/3171329995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36cc0fd840 0x7f36cc112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d1b01640 1 -- 172.21.15.43:0/3171329995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36cc0fd840 msgr2=0x7f36cc112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d1b01640 1 --2- 172.21.15.43:0/3171329995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36cc0fd840 0x7f36cc112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d1b01640 1 -- 172.21.15.43:0/3171329995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc0fe1e0 msgr2=0x7f36cc112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d1b01640 1 --2- 172.21.15.43:0/3171329995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc0fe1e0 0x7f36cc112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d1b01640 1 -- 172.21.15.43:0/3171329995 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c8005040 con 0x7f36cc106df0 2024-09-11T08:31:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.431+0000 7f36d0aff640 1 --2- 172.21.15.43:0/3171329995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36cc0fe1e0 0x7f36cc112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.432+0000 7f36d1b01640 1 --2- 172.21.15.43:0/3171329995 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36cc106df0 0x7f36cc10cc40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36c402a8d0 tx=0x7f36c4004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.432+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4038070 con 0x7f36cc106df0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.432+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c40098b0 con 0x7f36cc106df0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.432+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f36c4033e90 con 0x7f36cc106df0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.432+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c403db50 con 0x7f36cc106df0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.432+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc1198f0 con 0x7f36cc106df0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.433+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36cc05ddd0 con 0x7f36cc106df0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.434+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f36c4030070 con 0x7f36cc106df0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.434+0000 7f36c1ffb640 1 --2- 172.21.15.43:0/3171329995 >> v2:172.21.15.43:6800/588375333 conn(0x7f36b806ef80 0x7f36b8071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.435+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f36c40aec70 con 0x7f36cc106df0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.435+0000 7f36d1300640 1 --2- 172.21.15.43:0/3171329995 >> v2:172.21.15.43:6800/588375333 conn(0x7f36b806ef80 0x7f36b8071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.435+0000 7f36c1ffb640 1 --2- 172.21.15.43:0/3171329995 >> v2:172.21.15.172:6800/2140228578 conn(0x7f36b8078830 0x7f36b807ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.435+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 --> v2:172.21.15.172:6800/2140228578 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36b807b2f0 con 0x7f36b8078830 2024-09-11T08:31:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.435+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36c40b12b0 con 0x7f36cc106df0 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.435+0000 7f36d0aff640 1 --2- 172.21.15.43:0/3171329995 >> v2:172.21.15.172:6800/2140228578 conn(0x7f36b8078830 0x7f36b807ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.435+0000 7f36d1300640 1 --2- 172.21.15.43:0/3171329995 >> v2:172.21.15.43:6800/588375333 conn(0x7f36b806ef80 0x7f36b8071430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f36c8004a50 tx=0x7f36c800b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.435+0000 7f36d0aff640 1 --2- 172.21.15.43:0/3171329995 >> v2:172.21.15.172:6800/2140228578 conn(0x7f36b8078830 0x7f36b807ac60 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.436+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 <== osd.4 v2:172.21.15.172:6800/2140228578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f36b807b2f0 con 0x7f36b8078830 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.449+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 --> v2:172.21.15.172:6800/2140228578 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36cc066690 con 0x7f36b8078830 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36c1ffb640 1 -- 172.21.15.43:0/3171329995 <== osd.4 v2:172.21.15.172:6800/2140228578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f36cc066690 con 0x7f36b8078830 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 >> v2:172.21.15.172:6800/2140228578 conn(0x7f36b8078830 msgr2=0x7f36b807ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 --2- 172.21.15.43:0/3171329995 >> v2:172.21.15.172:6800/2140228578 conn(0x7f36b8078830 0x7f36b807ac60 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 >> v2:172.21.15.43:6800/588375333 conn(0x7f36b806ef80 msgr2=0x7f36b8071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 --2- 172.21.15.43:0/3171329995 >> v2:172.21.15.43:6800/588375333 conn(0x7f36b806ef80 0x7f36b8071430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f36c8004a50 tx=0x7f36c800b040 comp rx=0 tx=0).stop 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36cc106df0 msgr2=0x7f36cc10cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 --2- 172.21.15.43:0/3171329995 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36cc106df0 0x7f36cc10cc40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36c402a8d0 tx=0x7f36c4004d10 comp rx=0 tx=0).stop 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d1b01640 1 -- 172.21.15.43:0/3171329995 reap_dead start 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 shutdown_connections 2024-09-11T08:31:01.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 >> 172.21.15.43:0/3171329995 conn(0x7f36cc072580 msgr2=0x7f36cc0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:01.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 shutdown_connections 2024-09-11T08:31:01.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:01.450+0000 7f36d3564640 1 -- 172.21.15.43:0/3171329995 wait complete. 2024-09-11T08:31:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:01 smithi172 ceph-mon[32987]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:31:02.111 INFO:teuthology.orchestra.run.smithi043.stdout:55834574893 2024-09-11T08:31:02.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T08:31:03.077 INFO:teuthology.orchestra.run.smithi043.stdout:158913789978 2024-09-11T08:31:03.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T08:31:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:03.376 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:03 smithi043 ceph-mon[40592]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T08:31:03.376 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:03 smithi043 ceph-mon[29355]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T08:31:03.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:03 smithi172 ceph-mon[32987]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T08:31:03.761 INFO:teuthology.orchestra.run.smithi043.stdout:77309411367 2024-09-11T08:31:03.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T08:31:04.422 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:04.469 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:04.472 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:04.940 INFO:teuthology.orchestra.run.smithi043.stdout:133143986206 2024-09-11T08:31:04.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T08:31:05.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:05 smithi172 ceph-mon[32987]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:31:05.482 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:31:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:31:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:31:05.482 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:31:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:31:05.505 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:05 smithi043 ceph-mon[40592]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:31:05.505 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:05 smithi043 ceph-mon[29355]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:31:06.338 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:06 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:06.339 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:06 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:06 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:07.472 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:07 smithi043 ceph-mon[40592]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:07.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:07 smithi043 ceph-mon[29355]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:07 smithi172 ceph-mon[32987]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:09.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:09 smithi172 ceph-mon[32987]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:09 smithi043 ceph-mon[40592]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:09.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:09 smithi043 ceph-mon[29355]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:10.247 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:10.247 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:10.247 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:10.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 -- 172.21.15.43:0/3173910032 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c0fca60 msgr2=0x7ff92c0fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:10.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 --2- 172.21.15.43:0/3173910032 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c0fca60 0x7ff92c0fced0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff914004980 tx=0x7ff914009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 -- 172.21.15.43:0/3173910032 shutdown_connections 2024-09-11T08:31:10.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 --2- 172.21.15.43:0/3173910032 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c0fca60 0x7ff92c0fced0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 --2- 172.21.15.43:0/3173910032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0fd410 0x7ff92c1095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 --2- 172.21.15.43:0/3173910032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff92c1028a0 0x7ff92c102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 -- 172.21.15.43:0/3173910032 >> 172.21.15.43:0/3173910032 conn(0x7ff92c0738c0 msgr2=0x7ff92c073cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 -- 172.21.15.43:0/3173910032 shutdown_connections 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.501+0000 7ff9324e1640 1 -- 172.21.15.43:0/3173910032 wait complete. 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff9324e1640 1 Processor -- start 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff9324e1640 1 -- start start 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff9324e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff92c0fca60 0x7ff92c112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff9324e1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0fd410 0x7ff92c113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff9324e1640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c1028a0 0x7ff92c10cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff9324e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c10d430 con 0x7ff92c0fca60 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff9324e1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c10d5a0 con 0x7ff92c0fd410 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff9324e1640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c10d760 con 0x7ff92c1028a0 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff92bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff92c0fca60 0x7ff92c112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:10.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff930a7e640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c1028a0 0x7ff92c10cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff92bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff92c0fca60 0x7ff92c112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42468/0 (socket says 172.21.15.43:42468) 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.502+0000 7ff92bfff640 1 -- 172.21.15.43:0/2320554196 learned_addr learned my addr 172.21.15.43:0/2320554196 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff92b7fe640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0fd410 0x7ff92c113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff930a7e640 1 -- 172.21.15.43:0/2320554196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0fd410 msgr2=0x7ff92c113190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff930a7e640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0fd410 0x7ff92c113190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff930a7e640 1 -- 172.21.15.43:0/2320554196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff92c0fca60 msgr2=0x7ff92c112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff930a7e640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff92c0fca60 0x7ff92c112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff930a7e640 1 -- 172.21.15.43:0/2320554196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c005040 con 0x7ff92c1028a0 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff92bfff640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff92c0fca60 0x7ff92c112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff930a7e640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c1028a0 0x7ff92c10cde0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c910 tx=0x7ff924007cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff928ff9640 1 -- 172.21.15.43:0/2320554196 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400cea0 con 0x7ff92c1028a0 2024-09-11T08:31:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff928ff9640 1 -- 172.21.15.43:0/2320554196 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7ff92400acc0 con 0x7ff92c1028a0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff914005040 con 0x7ff92c1028a0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff928ff9640 1 -- 172.21.15.43:0/2320554196 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924012c50 con 0x7ff92c1028a0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.503+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c119aa0 con 0x7ff92c1028a0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.504+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c0fdfc0 con 0x7ff92c1028a0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.505+0000 7ff928ff9640 1 -- 172.21.15.43:0/2320554196 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7ff92400a850 con 0x7ff92c1028a0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.505+0000 7ff928ff9640 1 --2- 172.21.15.43:0/2320554196 >> v2:172.21.15.43:6800/588375333 conn(0x7ff91806ef80 0x7ff918071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.505+0000 7ff928ff9640 1 -- 172.21.15.43:0/2320554196 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7ff92408e9c0 con 0x7ff92c1028a0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.505+0000 7ff92bfff640 1 --2- 172.21.15.43:0/2320554196 >> v2:172.21.15.43:6800/588375333 conn(0x7ff91806ef80 0x7ff918071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.506+0000 7ff92bfff640 1 --2- 172.21.15.43:0/2320554196 >> v2:172.21.15.43:6800/588375333 conn(0x7ff91806ef80 0x7ff918071430 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff91c004f40 tx=0x7ff91c00b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.510+0000 7ff928ff9640 1 -- 172.21.15.43:0/2320554196 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff92405b820 con 0x7ff92c1028a0 2024-09-11T08:31:10.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.677+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff92c10bce0 con 0x7ff92c1028a0 2024-09-11T08:31:10.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.678+0000 7ff928ff9640 1 -- 172.21.15.43:0/2320554196 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff92405b1c0 con 0x7ff92c1028a0 2024-09-11T08:31:10.679 INFO:teuthology.orchestra.run.smithi043.stdout:55834574895 2024-09-11T08:31:10.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 >> v2:172.21.15.43:6800/588375333 conn(0x7ff91806ef80 msgr2=0x7ff918071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:10.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 --2- 172.21.15.43:0/2320554196 >> v2:172.21.15.43:6800/588375333 conn(0x7ff91806ef80 0x7ff918071430 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff91c004f40 tx=0x7ff91c00b040 comp rx=0 tx=0).stop 2024-09-11T08:31:10.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c1028a0 msgr2=0x7ff92c10cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:10.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c1028a0 0x7ff92c10cde0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c910 tx=0x7ff924007cf0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 shutdown_connections 2024-09-11T08:31:10.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 --2- 172.21.15.43:0/2320554196 >> v2:172.21.15.43:6800/588375333 conn(0x7ff91806ef80 0x7ff918071430 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff92c1028a0 0x7ff92c10cde0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0fd410 0x7ff92c113190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 --2- 172.21.15.43:0/2320554196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff92c0fca60 0x7ff92c112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 >> 172.21.15.43:0/2320554196 conn(0x7ff92c0738c0 msgr2=0x7ff92c0fac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 shutdown_connections 2024-09-11T08:31:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:10.680+0000 7ff9324e1640 1 -- 172.21.15.43:0/2320554196 wait complete. 2024-09-11T08:31:11.461 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:11 smithi043 ceph-mon[40592]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:11.461 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2320554196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:31:11.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:11 smithi043 ceph-mon[29355]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:11.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2320554196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:31:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:11 smithi172 ceph-mon[32987]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2320554196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:31:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 -- 172.21.15.43:0/3869752937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fc0a0 msgr2=0x7fec101084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 --2- 172.21.15.43:0/3869752937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fc0a0 0x7fec101084c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec08009d50 tx=0x7fec08004c50 comp rx=0 tx=0).stop 2024-09-11T08:31:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 -- 172.21.15.43:0/3869752937 shutdown_connections 2024-09-11T08:31:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 --2- 172.21.15.43:0/3869752937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fc0a0 0x7fec101084c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 --2- 172.21.15.43:0/3869752937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100fb6f0 0x7fec100fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 --2- 172.21.15.43:0/3869752937 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec10101710 0x7fec10101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 -- 172.21.15.43:0/3869752937 >> 172.21.15.43:0/3869752937 conn(0x7fec10071390 msgr2=0x7fec100f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 -- 172.21.15.43:0/3869752937 shutdown_connections 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.666+0000 7fec17b11640 1 -- 172.21.15.43:0/3869752937 wait complete. 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec17b11640 1 Processor -- start 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec17b11640 1 -- start start 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec17b11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fb6f0 0x7fec10112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec17b11640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100fc0a0 0x7fec10113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec17b11640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec10101710 0x7fec100650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec17b11640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec100656c0 con 0x7fec100fb6f0 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec17b11640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec10065830 con 0x7fec100fc0a0 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec17b11640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec100659f0 con 0x7fec10101710 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec158ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fb6f0 0x7fec10112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:11.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec158ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fb6f0 0x7fec10112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42508/0 (socket says 172.21.15.43:42508) 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec160ae640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec10101710 0x7fec100650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec158ad640 1 -- 172.21.15.43:0/426461492 learned_addr learned my addr 172.21.15.43:0/426461492 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.667+0000 7fec150ac640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100fc0a0 0x7fec10113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec158ad640 1 -- 172.21.15.43:0/426461492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec10101710 msgr2=0x7fec100650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec158ad640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec10101710 0x7fec100650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec158ad640 1 -- 172.21.15.43:0/426461492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100fc0a0 msgr2=0x7fec10113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec158ad640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100fc0a0 0x7fec10113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec158ad640 1 -- 172.21.15.43:0/426461492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c005040 con 0x7fec100fb6f0 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec150ac640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100fc0a0 0x7fec10113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec158ad640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fb6f0 0x7fec10112d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009550 tx=0x7fec0c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec067fc640 1 -- 172.21.15.43:0/426461492 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00e930 con 0x7fec100fb6f0 2024-09-11T08:31:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec067fc640 1 -- 172.21.15.43:0/426461492 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fec0c00f040 con 0x7fec100fb6f0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec080098b0 con 0x7fec100fb6f0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec067fc640 1 -- 172.21.15.43:0/426461492 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c013c60 con 0x7fec100fb6f0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.668+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec10119b50 con 0x7fec100fb6f0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.669+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec100fce30 con 0x7fec100fb6f0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.670+0000 7fec067fc640 1 -- 172.21.15.43:0/426461492 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fec0c00ea90 con 0x7fec100fb6f0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.670+0000 7fec067fc640 1 --2- 172.21.15.43:0/426461492 >> v2:172.21.15.43:6800/588375333 conn(0x7febfc06eed0 0x7febfc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.670+0000 7fec067fc640 1 -- 172.21.15.43:0/426461492 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7fec0c08e2e0 con 0x7fec100fb6f0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.670+0000 7fec150ac640 1 --2- 172.21.15.43:0/426461492 >> v2:172.21.15.43:6800/588375333 conn(0x7febfc06eed0 0x7febfc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.671+0000 7fec150ac640 1 --2- 172.21.15.43:0/426461492 >> v2:172.21.15.43:6800/588375333 conn(0x7febfc06eed0 0x7febfc071380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fec1010e5c0 tx=0x7fec00009040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.674+0000 7fec067fc640 1 -- 172.21.15.43:0/426461492 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fec0c05b1c0 con 0x7fec100fb6f0 2024-09-11T08:31:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 -- 172.21.15.43:0/509344195 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fc0e0 msgr2=0x7fe9a0108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 --2- 172.21.15.43:0/509344195 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fc0e0 0x7fe9a0108500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe998009d00 tx=0x7fe998004d10 comp rx=0 tx=0).stop 2024-09-11T08:31:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 -- 172.21.15.43:0/509344195 shutdown_connections 2024-09-11T08:31:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 --2- 172.21.15.43:0/509344195 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fc0e0 0x7fe9a0108500 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 --2- 172.21.15.43:0/509344195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a00fb730 0x7fe9a00fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 --2- 172.21.15.43:0/509344195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a0101750 0x7fe9a0101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 -- 172.21.15.43:0/509344195 >> 172.21.15.43:0/509344195 conn(0x7fe9a0072580 msgr2=0x7fe9a00f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 -- 172.21.15.43:0/509344195 shutdown_connections 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.544+0000 7fe9a788e640 1 -- 172.21.15.43:0/509344195 wait complete. 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a788e640 1 Processor -- start 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a788e640 1 -- start start 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a788e640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fb730 0x7fe9a0112ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a788e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a00fc0e0 0x7fe9a0113000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a788e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a0101750 0x7fe9a010cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a788e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a010d2a0 con 0x7fe9a00fc0e0 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a788e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a010d410 con 0x7fe9a0101750 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a788e640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a010d5d0 con 0x7fe9a00fb730 2024-09-11T08:31:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a562a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fb730 0x7fe9a0112ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a4e29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a00fc0e0 0x7fe9a0113000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a562a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fb730 0x7fe9a0112ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48006/0 (socket says 172.21.15.43:48006) 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a4e29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a00fc0e0 0x7fe9a0113000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42494/0 (socket says 172.21.15.43:42494) 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.545+0000 7fe9a562a640 1 -- 172.21.15.43:0/3802542708 learned_addr learned my addr 172.21.15.43:0/3802542708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a5e2b640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a0101750 0x7fe9a010cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a562a640 1 -- 172.21.15.43:0/3802542708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a0101750 msgr2=0x7fe9a010cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a562a640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a0101750 0x7fe9a010cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a562a640 1 -- 172.21.15.43:0/3802542708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a00fc0e0 msgr2=0x7fe9a0113000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a562a640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a00fc0e0 0x7fe9a0113000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a562a640 1 -- 172.21.15.43:0/3802542708 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe990005040 con 0x7fe9a00fb730 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a4e29640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a00fc0e0 0x7fe9a0113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a562a640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fb730 0x7fe9a0112ac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c8b0 tx=0x7fe99c004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe995ffb640 1 -- 172.21.15.43:0/3802542708 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c00a640 con 0x7fe9a00fb730 2024-09-11T08:31:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe995ffb640 1 -- 172.21.15.43:0/3802542708 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fe99c00ac10 con 0x7fe9a00fb730 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9980098b0 con 0x7fe9a00fb730 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe995ffb640 1 -- 172.21.15.43:0/3802542708 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c012b50 con 0x7fe9a00fb730 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.546+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a0119910 con 0x7fe9a00fb730 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.547+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a00fce70 con 0x7fe9a00fb730 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.548+0000 7fe995ffb640 1 -- 172.21.15.43:0/3802542708 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fe99c00a7a0 con 0x7fe9a00fb730 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.548+0000 7fe995ffb640 1 --2- 172.21.15.43:0/3802542708 >> v2:172.21.15.43:6800/588375333 conn(0x7fe98c06ef80 0x7fe98c071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.548+0000 7fe995ffb640 1 -- 172.21.15.43:0/3802542708 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7fe99c08e620 con 0x7fe9a00fb730 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.548+0000 7fe9a4e29640 1 --2- 172.21.15.43:0/3802542708 >> v2:172.21.15.43:6800/588375333 conn(0x7fe98c06ef80 0x7fe98c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.549+0000 7fe9a4e29640 1 --2- 172.21.15.43:0/3802542708 >> v2:172.21.15.43:6800/588375333 conn(0x7fe98c06ef80 0x7fe98c071430 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9a010e380 tx=0x7fe99000b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.553+0000 7fe995ffb640 1 -- 172.21.15.43:0/3802542708 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe99c05b5e0 con 0x7fe9a00fb730 2024-09-11T08:31:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.721+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe9a010ab90 con 0x7fe9a00fb730 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stdout:158913789981 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.722+0000 7fe995ffb640 1 -- 172.21.15.43:0/3802542708 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe99c05af80 con 0x7fe9a00fb730 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 >> v2:172.21.15.43:6800/588375333 conn(0x7fe98c06ef80 msgr2=0x7fe98c071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 --2- 172.21.15.43:0/3802542708 >> v2:172.21.15.43:6800/588375333 conn(0x7fe98c06ef80 0x7fe98c071430 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9a010e380 tx=0x7fe99000b040 comp rx=0 tx=0).stop 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fb730 msgr2=0x7fe9a0112ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fb730 0x7fe9a0112ac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c8b0 tx=0x7fe99c004430 comp rx=0 tx=0).stop 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 shutdown_connections 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 --2- 172.21.15.43:0/3802542708 >> v2:172.21.15.43:6800/588375333 conn(0x7fe98c06ef80 0x7fe98c071430 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a0101750 0x7fe9a010cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a00fc0e0 0x7fe9a0113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 --2- 172.21.15.43:0/3802542708 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9a00fb730 0x7fe9a0112ac0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 >> 172.21.15.43:0/3802542708 conn(0x7fe9a0072580 msgr2=0x7fe9a0066740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 shutdown_connections 2024-09-11T08:31:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.724+0000 7fe9a788e640 1 -- 172.21.15.43:0/3802542708 wait complete. 2024-09-11T08:31:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.843+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fec1010aaf0 con 0x7fec100fb6f0 2024-09-11T08:31:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.844+0000 7fec067fc640 1 -- 172.21.15.43:0/426461492 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec0c05ab60 con 0x7fec100fb6f0 2024-09-11T08:31:11.844 INFO:teuthology.orchestra.run.smithi043.stdout:210453397522 2024-09-11T08:31:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.845+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 >> v2:172.21.15.43:6800/588375333 conn(0x7febfc06eed0 msgr2=0x7febfc071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.845+0000 7fec17b11640 1 --2- 172.21.15.43:0/426461492 >> v2:172.21.15.43:6800/588375333 conn(0x7febfc06eed0 0x7febfc071380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fec1010e5c0 tx=0x7fec00009040 comp rx=0 tx=0).stop 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.845+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fb6f0 msgr2=0x7fec10112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.845+0000 7fec17b11640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fb6f0 0x7fec10112d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009550 tx=0x7fec0c009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.845+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 shutdown_connections 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.845+0000 7fec17b11640 1 --2- 172.21.15.43:0/426461492 >> v2:172.21.15.43:6800/588375333 conn(0x7febfc06eed0 0x7febfc071380 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.845+0000 7fec17b11640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec10101710 0x7fec100650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.845+0000 7fec17b11640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100fc0a0 0x7fec10113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.846+0000 7fec17b11640 1 --2- 172.21.15.43:0/426461492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec100fb6f0 0x7fec10112d30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.846+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 >> 172.21.15.43:0/426461492 conn(0x7fec10071390 msgr2=0x7fec10072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.846+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 shutdown_connections 2024-09-11T08:31:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:11.846+0000 7fec17b11640 1 -- 172.21.15.43:0/426461492 wait complete. 2024-09-11T08:31:12.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3802542708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:31:12.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/426461492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:31:12.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:12.353 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3802542708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:31:12.353 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/426461492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:31:12.353 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3802542708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:31:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/426461492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:31:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:12.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574895 for osd.1 2024-09-11T08:31:12.670 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:13.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:13 smithi043 ceph-mon[40592]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:13.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:13 smithi043 ceph-mon[29355]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:13 smithi172 ceph-mon[32987]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:14.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397518 got 210453397522 for osd.7 2024-09-11T08:31:14.194 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:14.614 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789978 got 158913789981 for osd.5 2024-09-11T08:31:14.615 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:15.473 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:15 smithi043 ceph-mon[40592]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:15.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:15 smithi043 ceph-mon[29355]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:15 smithi172 ceph-mon[32987]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:15.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:31:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:31:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:31:15.733 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:31:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:31:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 -- 172.21.15.43:0/3071328644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fd840 msgr2=0x7f06640fdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 --2- 172.21.15.43:0/3071328644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fd840 0x7f06640fdc10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0654009cd0 tx=0x7f0654004d10 comp rx=0 tx=0).stop 2024-09-11T08:31:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 -- 172.21.15.43:0/3071328644 shutdown_connections 2024-09-11T08:31:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 --2- 172.21.15.43:0/3071328644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0664106df0 0x7f06641091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 --2- 172.21.15.43:0/3071328644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06640fe1e0 0x7f06641068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 --2- 172.21.15.43:0/3071328644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fd840 0x7f06640fdc10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 -- 172.21.15.43:0/3071328644 >> 172.21.15.43:0/3071328644 conn(0x7f0664072580 msgr2=0x7f06640f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:15.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 -- 172.21.15.43:0/3071328644 shutdown_connections 2024-09-11T08:31:15.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.679+0000 7f0669eac640 1 -- 172.21.15.43:0/3071328644 wait complete. 2024-09-11T08:31:15.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0669eac640 1 Processor -- start 2024-09-11T08:31:15.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0669eac640 1 -- start start 2024-09-11T08:31:15.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0669eac640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06640fd840 0x7f06640fb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0669eac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fe1e0 0x7f06640fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0669eac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0664106df0 0x7f06640fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0669eac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066411b360 con 0x7f06640fe1e0 2024-09-11T08:31:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0669eac640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066411b4d0 con 0x7f0664106df0 2024-09-11T08:31:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0669eac640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066411b690 con 0x7f06640fd840 2024-09-11T08:31:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f06637fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06640fd840 0x7f06640fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:15.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fe1e0 0x7f06640fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:15.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f06637fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06640fd840 0x7f06640fb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:57388/0 (socket says 172.21.15.43:57388) 2024-09-11T08:31:15.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f06637fe640 1 -- 172.21.15.43:0/606395004 learned_addr learned my addr 172.21.15.43:0/606395004 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:15.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.680+0000 7f0663fff640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0664106df0 0x7f06640fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:15.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0662ffd640 1 -- 172.21.15.43:0/606395004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06640fd840 msgr2=0x7f06640fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:15.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0662ffd640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06640fd840 0x7f06640fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0662ffd640 1 -- 172.21.15.43:0/606395004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0664106df0 msgr2=0x7f06640fc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0662ffd640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0664106df0 0x7f06640fc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0662ffd640 1 -- 172.21.15.43:0/606395004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c005040 con 0x7f06640fe1e0 2024-09-11T08:31:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f06637fe640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06640fd840 0x7f06640fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0663fff640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0664106df0 0x7f06640fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T08:31:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0662ffd640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fe1e0 0x7f06640fbb70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f065800c760 tx=0x7f0658004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f064bfff640 1 -- 172.21.15.43:0/606395004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065800a5b0 con 0x7f06640fe1e0 2024-09-11T08:31:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f064bfff640 1 -- 172.21.15.43:0/606395004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f065800ab80 con 0x7f06640fe1e0 2024-09-11T08:31:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06540098b0 con 0x7f06640fe1e0 2024-09-11T08:31:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f064bfff640 1 -- 172.21.15.43:0/606395004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658012af0 con 0x7f06640fe1e0 2024-09-11T08:31:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.681+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066411bd00 con 0x7f06640fe1e0 2024-09-11T08:31:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.682+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0664066690 con 0x7f06640fe1e0 2024-09-11T08:31:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.682+0000 7f064bfff640 1 -- 172.21.15.43:0/606395004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f065800acf0 con 0x7f06640fe1e0 2024-09-11T08:31:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.683+0000 7f064bfff640 1 --2- 172.21.15.43:0/606395004 >> v2:172.21.15.43:6800/588375333 conn(0x7f064c06efb0 0x7f064c071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.683+0000 7f064bfff640 1 -- 172.21.15.43:0/606395004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f065808e2c0 con 0x7f06640fe1e0 2024-09-11T08:31:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.683+0000 7f06637fe640 1 --2- 172.21.15.43:0/606395004 >> v2:172.21.15.43:6800/588375333 conn(0x7f064c06efb0 0x7f064c071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.684+0000 7f06637fe640 1 --2- 172.21.15.43:0/606395004 >> v2:172.21.15.43:6800/588375333 conn(0x7f064c06efb0 0x7f064c071460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f065402afa0 tx=0x7f0654035040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:15.952 INFO:teuthology.orchestra.run.smithi043.stdout:77309411370 2024-09-11T08:31:15.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.687+0000 7f064bfff640 1 -- 172.21.15.43:0/606395004 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f065805b340 con 0x7f06640fe1e0 2024-09-11T08:31:15.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.855+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f06640fcb10 con 0x7f06640fe1e0 2024-09-11T08:31:15.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.855+0000 7f064bfff640 1 -- 172.21.15.43:0/606395004 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f065805ace0 con 0x7f06640fe1e0 2024-09-11T08:31:15.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 >> v2:172.21.15.43:6800/588375333 conn(0x7f064c06efb0 msgr2=0x7f064c071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 --2- 172.21.15.43:0/606395004 >> v2:172.21.15.43:6800/588375333 conn(0x7f064c06efb0 0x7f064c071460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f065402afa0 tx=0x7f0654035040 comp rx=0 tx=0).stop 2024-09-11T08:31:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fe1e0 msgr2=0x7f06640fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fe1e0 0x7f06640fbb70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f065800c760 tx=0x7f0658004330 comp rx=0 tx=0).stop 2024-09-11T08:31:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 shutdown_connections 2024-09-11T08:31:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 --2- 172.21.15.43:0/606395004 >> v2:172.21.15.43:6800/588375333 conn(0x7f064c06efb0 0x7f064c071460 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0664106df0 0x7f06640fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06640fe1e0 0x7f06640fbb70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 --2- 172.21.15.43:0/606395004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06640fd840 0x7f06640fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 >> 172.21.15.43:0/606395004 conn(0x7f0664072580 msgr2=0x7f06640fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.857+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 shutdown_connections 2024-09-11T08:31:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:15.858+0000 7f0669eac640 1 -- 172.21.15.43:0/606395004 wait complete. 2024-09-11T08:31:16.558 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:16 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:16.559 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/606395004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:31:16.559 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:16 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:16.559 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/606395004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:31:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:16 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/606395004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:31:17.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:17 smithi043 ceph-mon[29355]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:17.598 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:17 smithi043 ceph-mon[40592]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:17 smithi172 ceph-mon[32987]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:19.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:19 smithi043 ceph-mon[29355]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:19.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:19 smithi043 ceph-mon[40592]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:19.499 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411367 got 77309411370 for osd.2 2024-09-11T08:31:19.499 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:19.533 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:19 smithi172 ceph-mon[32987]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:20.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 -- 172.21.15.43:0/3131504670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 msgr2=0x7f025406ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 --2- 172.21.15.43:0/3131504670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 0x7f025406ff10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0244004950 tx=0x7f0244009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 -- 172.21.15.43:0/3131504670 shutdown_connections 2024-09-11T08:31:20.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 --2- 172.21.15.43:0/3131504670 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0254109050 0x7f025410b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 --2- 172.21.15.43:0/3131504670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02540704e0 0x7f0254108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 --2- 172.21.15.43:0/3131504670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 0x7f025406ff10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 -- 172.21.15.43:0/3131504670 >> 172.21.15.43:0/3131504670 conn(0x7f0254073550 msgr2=0x7f02540faa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 -- 172.21.15.43:0/3131504670 shutdown_connections 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.222+0000 7f025a64c640 1 -- 172.21.15.43:0/3131504670 wait complete. 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f025a64c640 1 Processor -- start 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f025a64c640 1 -- start start 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f025a64c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 0x7f0254115330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f025a64c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02540704e0 0x7f0254115870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f025a64c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0254109050 0x7f025410f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f025a64c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025410f900 con 0x7f025406fb40 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f025a64c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025410fa70 con 0x7f0254109050 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f025a64c640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025410fc30 con 0x7f02540704e0 2024-09-11T08:31:20.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f0253fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 0x7f0254115330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f02537fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02540704e0 0x7f0254115870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f0253fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 0x7f0254115330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58450/0 (socket says 172.21.15.43:58450) 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.223+0000 7f0253fff640 1 -- 172.21.15.43:0/3333166702 learned_addr learned my addr 172.21.15.43:0/3333166702 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0253fff640 1 -- 172.21.15.43:0/3333166702 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02540704e0 msgr2=0x7f0254115870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0253fff640 1 --2- 172.21.15.43:0/3333166702 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02540704e0 0x7f0254115870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0253fff640 1 -- 172.21.15.43:0/3333166702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0254109050 msgr2=0x7f025410f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0258be9640 1 --2- 172.21.15.43:0/3333166702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0254109050 0x7f025410f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0253fff640 1 --2- 172.21.15.43:0/3333166702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0254109050 0x7f025410f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0253fff640 1 -- 172.21.15.43:0/3333166702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0098b0 con 0x7f025406fb40 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0253fff640 1 --2- 172.21.15.43:0/3333166702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 0x7f0254115330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02440049b0 tx=0x7f0244009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0250ff9640 1 -- 172.21.15.43:0/3333166702 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024402a960 con 0x7f025406fb40 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0250ff9640 1 -- 172.21.15.43:0/3333166702 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f02440325e0 con 0x7f025406fb40 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0244005040 con 0x7f025406fb40 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f0250ff9640 1 -- 172.21.15.43:0/3333166702 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024403c690 con 0x7f025406fb40 2024-09-11T08:31:20.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.224+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025411c180 con 0x7f025406fb40 2024-09-11T08:31:20.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.225+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025405ed70 con 0x7f025406fb40 2024-09-11T08:31:20.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.226+0000 7f0250ff9640 1 -- 172.21.15.43:0/3333166702 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f0244030020 con 0x7f025406fb40 2024-09-11T08:31:20.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.226+0000 7f0250ff9640 1 --2- 172.21.15.43:0/3333166702 >> v2:172.21.15.43:6800/588375333 conn(0x7f023c06eed0 0x7f023c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:20.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.227+0000 7f02537fe640 1 --2- 172.21.15.43:0/3333166702 >> v2:172.21.15.43:6800/588375333 conn(0x7f023c06eed0 0x7f023c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:20.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.227+0000 7f0250ff9640 1 -- 172.21.15.43:0/3333166702 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f02440adff0 con 0x7f025406fb40 2024-09-11T08:31:20.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.227+0000 7f02537fe640 1 --2- 172.21.15.43:0/3333166702 >> v2:172.21.15.43:6800/588375333 conn(0x7f023c06eed0 0x7f023c071380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0254110bf0 tx=0x7f0248009040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:20.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.230+0000 7f0250ff9640 1 -- 172.21.15.43:0/3333166702 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f024407afb0 con 0x7f025406fb40 2024-09-11T08:31:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.135+0000 7ff62ae3d640 1 -- 172.21.15.43:0/4184016925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6240fd1c0 msgr2=0x7ff6241095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.135+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/4184016925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6240fd1c0 0x7ff6241095e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009d00 tx=0x7ff61c004d10 comp rx=0 tx=0).stop 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.135+0000 7ff62ae3d640 1 -- 172.21.15.43:0/4184016925 shutdown_connections 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.135+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/4184016925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6240fd1c0 0x7ff6241095e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.135+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/4184016925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6240fc810 0x7ff6240fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.135+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/4184016925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff624102830 0x7ff624102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.135+0000 7ff62ae3d640 1 -- 172.21.15.43:0/4184016925 >> 172.21.15.43:0/4184016925 conn(0x7ff624073590 msgr2=0x7ff6240faa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.136+0000 7ff62ae3d640 1 -- 172.21.15.43:0/4184016925 shutdown_connections 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.136+0000 7ff62ae3d640 1 -- 172.21.15.43:0/4184016925 wait complete. 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.136+0000 7ff62ae3d640 1 Processor -- start 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.136+0000 7ff62ae3d640 1 -- start start 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.136+0000 7ff62ae3d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6240fc810 0x7ff624112a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.136+0000 7ff62ae3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6240fd1c0 0x7ff624112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff62ae3d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff624102830 0x7ff62410cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff62ae3d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62410d220 con 0x7ff624102830 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff62ae3d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62410d390 con 0x7ff6240fd1c0 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff62ae3d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62410d550 con 0x7ff6240fc810 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6240fc810 0x7ff624112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6240fc810 0x7ff624112a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:57400/0 (socket says 172.21.15.43:57400) 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff6293da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff624102830 0x7ff62410cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 -- 172.21.15.43:0/2947289521 learned_addr learned my addr 172.21.15.43:0/2947289521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff623fff640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6240fd1c0 0x7ff624112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 -- 172.21.15.43:0/2947289521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6240fd1c0 msgr2=0x7ff624112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6240fd1c0 0x7ff624112f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 -- 172.21.15.43:0/2947289521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff624102830 msgr2=0x7ff62410cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff624102830 0x7ff62410cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 -- 172.21.15.43:0/2947289521 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618005040 con 0x7ff6240fc810 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff623fff640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6240fd1c0 0x7ff624112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff6293da640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff624102830 0x7ff62410cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff628bd9640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6240fc810 0x7ff624112a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff618009780 tx=0x7ff618009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff6217fa640 1 -- 172.21.15.43:0/2947289521 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180099c0 con 0x7ff6240fc810 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff6217fa640 1 -- 172.21.15.43:0/2947289521 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7ff61800f040 con 0x7ff6240fc810 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff6217fa640 1 -- 172.21.15.43:0/2947289521 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618013ca0 con 0x7ff6240fc810 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.137+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c0098b0 con 0x7ff6240fc810 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.138+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff624119890 con 0x7ff6240fc810 2024-09-11T08:31:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.138+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6240fdf50 con 0x7ff6240fc810 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.139+0000 7ff6217fa640 1 -- 172.21.15.43:0/2947289521 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7ff61800ebb0 con 0x7ff6240fc810 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.140+0000 7ff6217fa640 1 --2- 172.21.15.43:0/2947289521 >> v2:172.21.15.43:6800/588375333 conn(0x7ff61406ef80 0x7ff614071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.140+0000 7ff6217fa640 1 -- 172.21.15.43:0/2947289521 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7ff61808e3a0 con 0x7ff6240fc810 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.140+0000 7ff623fff640 1 --2- 172.21.15.43:0/2947289521 >> v2:172.21.15.43:6800/588375333 conn(0x7ff61406ef80 0x7ff614071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.140+0000 7ff623fff640 1 --2- 172.21.15.43:0/2947289521 >> v2:172.21.15.43:6800/588375333 conn(0x7ff61406ef80 0x7ff614071430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff610009fd0 tx=0x7ff610009e50 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.144+0000 7ff6217fa640 1 -- 172.21.15.43:0/2947289521 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff61805b280 con 0x7ff6240fc810 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.316+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff62410bd00 con 0x7ff6240fc810 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.316+0000 7ff6217fa640 1 -- 172.21.15.43:0/2947289521 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff618013e00 con 0x7ff6240fc810 2024-09-11T08:31:20.338 INFO:teuthology.orchestra.run.smithi043.stdout:34359738420 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 >> v2:172.21.15.43:6800/588375333 conn(0x7ff61406ef80 msgr2=0x7ff614071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/2947289521 >> v2:172.21.15.43:6800/588375333 conn(0x7ff61406ef80 0x7ff614071430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff610009fd0 tx=0x7ff610009e50 comp rx=0 tx=0).stop 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6240fc810 msgr2=0x7ff624112a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6240fc810 0x7ff624112a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff618009780 tx=0x7ff618009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 shutdown_connections 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/2947289521 >> v2:172.21.15.43:6800/588375333 conn(0x7ff61406ef80 0x7ff614071430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff624102830 0x7ff62410cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6240fd1c0 0x7ff624112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 --2- 172.21.15.43:0/2947289521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6240fc810 0x7ff624112a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 >> 172.21.15.43:0/2947289521 conn(0x7ff624073590 msgr2=0x7ff6240fb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 shutdown_connections 2024-09-11T08:31:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.318+0000 7ff62ae3d640 1 -- 172.21.15.43:0/2947289521 wait complete. 2024-09-11T08:31:20.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.401+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f025404b750 con 0x7f025406fb40 2024-09-11T08:31:20.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.401+0000 7f0250ff9640 1 -- 172.21.15.43:0/3333166702 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0244080090 con 0x7f025406fb40 2024-09-11T08:31:20.401 INFO:teuthology.orchestra.run.smithi043.stdout:184683593753 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 >> v2:172.21.15.43:6800/588375333 conn(0x7f023c06eed0 msgr2=0x7f023c071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 --2- 172.21.15.43:0/3333166702 >> v2:172.21.15.43:6800/588375333 conn(0x7f023c06eed0 0x7f023c071380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0254110bf0 tx=0x7f0248009040 comp rx=0 tx=0).stop 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 msgr2=0x7f0254115330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 --2- 172.21.15.43:0/3333166702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 0x7f0254115330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02440049b0 tx=0x7f0244009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 shutdown_connections 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 --2- 172.21.15.43:0/3333166702 >> v2:172.21.15.43:6800/588375333 conn(0x7f023c06eed0 0x7f023c071380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 --2- 172.21.15.43:0/3333166702 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02540704e0 0x7f0254115870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 --2- 172.21.15.43:0/3333166702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0254109050 0x7f025410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 --2- 172.21.15.43:0/3333166702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f025406fb40 0x7f0254115330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 >> 172.21.15.43:0/3333166702 conn(0x7f0254073550 msgr2=0x7f0254100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 shutdown_connections 2024-09-11T08:31:20.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:20.403+0000 7f025a64c640 1 -- 172.21.15.43:0/3333166702 wait complete. 2024-09-11T08:31:21.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738416 got 34359738420 for osd.0 2024-09-11T08:31:21.321 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:21.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:21 smithi043 ceph-mon[40592]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:21.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2947289521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:31:21.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3333166702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:31:21.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:21 smithi043 ceph-mon[29355]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:21.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2947289521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:31:21.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3333166702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:31:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:21 smithi172 ceph-mon[32987]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2947289521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:31:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3333166702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:31:22.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593748 got 184683593753 for osd.6 2024-09-11T08:31:22.823 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:23.365 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:23 smithi043 ceph-mon[40592]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:23.366 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:23 smithi043 ceph-mon[29355]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:23.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.455+0000 7f8e57ed4640 1 -- 172.21.15.43:0/3687027111 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 msgr2=0x7f8e500fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.455+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/3687027111 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 0x7f8e500fbd40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e40004980 tx=0x7f8e40009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.455+0000 7f8e57ed4640 1 -- 172.21.15.43:0/3687027111 shutdown_connections 2024-09-11T08:31:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.455+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/3687027111 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 0x7f8e500fbd40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.455+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/3687027111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e500fc280 0x7f8e50108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.455+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/3687027111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e501017b0 0x7f8e50101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.455+0000 7f8e57ed4640 1 -- 172.21.15.43:0/3687027111 >> 172.21.15.43:0/3687027111 conn(0x7f8e50072630 msgr2=0x7f8e50072a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.456+0000 7f8e57ed4640 1 -- 172.21.15.43:0/3687027111 shutdown_connections 2024-09-11T08:31:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.456+0000 7f8e57ed4640 1 -- 172.21.15.43:0/3687027111 wait complete. 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.456+0000 7f8e57ed4640 1 Processor -- start 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.456+0000 7f8e57ed4640 1 -- start start 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.456+0000 7f8e57ed4640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 0x7f8e50112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.456+0000 7f8e57ed4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e500fc280 0x7f8e50113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e57ed4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e501017b0 0x7f8e5010cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e57ed4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5010d3c0 con 0x7f8e500fc280 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e57ed4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5010d530 con 0x7f8e501017b0 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e57ed4640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5010d6f0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e5546f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e500fc280 0x7f8e50113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e55c70640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 0x7f8e50112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e5546f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e500fc280 0x7f8e50113160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53488/0 (socket says 172.21.15.43:53488) 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e5546f640 1 -- 172.21.15.43:0/2274813072 learned_addr learned my addr 172.21.15.43:0/2274813072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e56471640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e501017b0 0x7f8e5010cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e55c70640 1 -- 172.21.15.43:0/2274813072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e501017b0 msgr2=0x7f8e5010cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e55c70640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e501017b0 0x7f8e5010cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e55c70640 1 -- 172.21.15.43:0/2274813072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e500fc280 msgr2=0x7f8e50113160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e55c70640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e500fc280 0x7f8e50113160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e55c70640 1 -- 172.21.15.43:0/2274813072 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480098b0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e5546f640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e500fc280 0x7f8e50113160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:23.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e55c70640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 0x7f8e50112c20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00c760 tx=0x7f8e4c004330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:23.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e467fc640 1 -- 172.21.15.43:0/2274813072 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00a5b0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.457+0000 7f8e467fc640 1 -- 172.21.15.43:0/2274813072 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8e4c00ab80 con 0x7f8e500fb8f0 2024-09-11T08:31:23.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.458+0000 7f8e467fc640 1 -- 172.21.15.43:0/2274813072 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c012af0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.458+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e40005040 con 0x7f8e500fb8f0 2024-09-11T08:31:23.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.458+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e50119ab0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.458+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e500fced0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.459+0000 7f8e467fc640 1 -- 172.21.15.43:0/2274813072 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f8e4c00acf0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.460+0000 7f8e467fc640 1 --2- 172.21.15.43:0/2274813072 >> v2:172.21.15.43:6800/588375333 conn(0x7f8e3c06ef80 0x7f8e3c071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.460+0000 7f8e467fc640 1 -- 172.21.15.43:0/2274813072 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f8e4c08e1b0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.460+0000 7f8e5546f640 1 --2- 172.21.15.43:0/2274813072 >> v2:172.21.15.43:6800/588375333 conn(0x7f8e3c06ef80 0x7f8e3c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.460+0000 7f8e5546f640 1 --2- 172.21.15.43:0/2274813072 >> v2:172.21.15.43:6800/588375333 conn(0x7f8e3c06ef80 0x7f8e3c071430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e40007ac0 tx=0x7f8e40009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.464+0000 7f8e467fc640 1 -- 172.21.15.43:0/2274813072 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8e4c05b220 con 0x7f8e500fb8f0 2024-09-11T08:31:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.634+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8e5010abd0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.634+0000 7f8e467fc640 1 -- 172.21.15.43:0/2274813072 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e4c05abc0 con 0x7f8e500fb8f0 2024-09-11T08:31:23.634 INFO:teuthology.orchestra.run.smithi043.stdout:133143986210 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.636+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 >> v2:172.21.15.43:6800/588375333 conn(0x7f8e3c06ef80 msgr2=0x7f8e3c071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.636+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/2274813072 >> v2:172.21.15.43:6800/588375333 conn(0x7f8e3c06ef80 0x7f8e3c071430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e40007ac0 tx=0x7f8e40009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.636+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 msgr2=0x7f8e50112c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.636+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 0x7f8e50112c20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00c760 tx=0x7f8e4c004330 comp rx=0 tx=0).stop 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.636+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 shutdown_connections 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.637+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/2274813072 >> v2:172.21.15.43:6800/588375333 conn(0x7f8e3c06ef80 0x7f8e3c071430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.637+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e501017b0 0x7f8e5010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.637+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e500fc280 0x7f8e50113160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.637+0000 7f8e57ed4640 1 --2- 172.21.15.43:0/2274813072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8e500fb8f0 0x7f8e50112c20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.637+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 >> 172.21.15.43:0/2274813072 conn(0x7f8e50072630 msgr2=0x7f8e500fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.637+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 shutdown_connections 2024-09-11T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:23.637+0000 7f8e57ed4640 1 -- 172.21.15.43:0/2274813072 wait complete. 2024-09-11T08:31:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:23 smithi172 ceph-mon[32987]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2274813072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:31:24.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2274813072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:31:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2274813072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:31:25.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:25 smithi043 ceph-mon[29355]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:25.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:25 smithi043 ceph-mon[40592]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:25.722 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986206 got 133143986210 for osd.4 2024-09-11T08:31:25.722 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:25 smithi172 ceph-mon[32987]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:31:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:31:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:31:25.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:31:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:31:25.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 -- 172.21.15.43:0/4103056760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6138103190 msgr2=0x7f61380facf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:25.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 --2- 172.21.15.43:0/4103056760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6138103190 0x7f61380facf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6128009d20 tx=0x7f6128004c50 comp rx=0 tx=0).stop 2024-09-11T08:31:25.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 -- 172.21.15.43:0/4103056760 shutdown_connections 2024-09-11T08:31:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 --2- 172.21.15.43:0/4103056760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6138103190 0x7f61380facf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 --2- 172.21.15.43:0/4103056760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61380fb420 0x7f61381095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 --2- 172.21.15.43:0/4103056760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61381027f0 0x7f6138102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 -- 172.21.15.43:0/4103056760 >> 172.21.15.43:0/4103056760 conn(0x7f6138074d00 msgr2=0x7f61380f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 -- 172.21.15.43:0/4103056760 shutdown_connections 2024-09-11T08:31:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.896+0000 7f613ebf1640 1 -- 172.21.15.43:0/4103056760 wait complete. 2024-09-11T08:31:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613ebf1640 1 Processor -- start 2024-09-11T08:31:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613ebf1640 1 -- start start 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613ebf1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61380fb420 0x7f6138113090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613ebf1640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61381027f0 0x7f61381135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613ebf1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103190 0x7f613810d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613ebf1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613810d660 con 0x7f61380fb420 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613ebf1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613810d7d0 con 0x7f6138103190 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613ebf1640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613810d990 con 0x7f61381027f0 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613c98d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61380fb420 0x7f6138113090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613c98d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61380fb420 0x7f6138113090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53514/0 (socket says 172.21.15.43:53514) 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f613c98d640 1 -- 172.21.15.43:0/2383892001 learned_addr learned my addr 172.21.15.43:0/2383892001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.897+0000 7f6137fff640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61381027f0 0x7f61381135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613d18e640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103190 0x7f613810d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613c98d640 1 -- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61381027f0 msgr2=0x7f61381135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613c98d640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61381027f0 0x7f61381135d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613c98d640 1 -- 172.21.15.43:0/2383892001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103190 msgr2=0x7f613810d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613c98d640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103190 0x7f613810d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613c98d640 1 -- 172.21.15.43:0/2383892001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6130005040 con 0x7f61380fb420 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f6137fff640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61381027f0 0x7f61381135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613d18e640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103190 0x7f613810d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613c98d640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61380fb420 0x7f6138113090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f612000b810 tx=0x7f61200091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f61357fa640 1 -- 172.21.15.43:0/2383892001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612000be60 con 0x7f61380fb420 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f61357fa640 1 -- 172.21.15.43:0/2383892001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f612000cba0 con 0x7f61380fb420 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61280098b0 con 0x7f61380fb420 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f61357fa640 1 -- 172.21.15.43:0/2383892001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120015b60 con 0x7f61380fb420 2024-09-11T08:31:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.898+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6138119ee0 con 0x7f61380fb420 2024-09-11T08:31:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.899+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61380fe010 con 0x7f61380fb420 2024-09-11T08:31:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.900+0000 7f61357fa640 1 -- 172.21.15.43:0/2383892001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f6120012070 con 0x7f61380fb420 2024-09-11T08:31:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.900+0000 7f61357fa640 1 --2- 172.21.15.43:0/2383892001 >> v2:172.21.15.43:6800/588375333 conn(0x7f612406eec0 0x7f6124071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.901+0000 7f61357fa640 1 -- 172.21.15.43:0/2383892001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f612008dc20 con 0x7f61380fb420 2024-09-11T08:31:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.901+0000 7f6137fff640 1 --2- 172.21.15.43:0/2383892001 >> v2:172.21.15.43:6800/588375333 conn(0x7f612406eec0 0x7f6124071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.901+0000 7f6137fff640 1 --2- 172.21.15.43:0/2383892001 >> v2:172.21.15.43:6800/588375333 conn(0x7f612406eec0 0x7f6124071370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6128009cf0 tx=0x7f6128035040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:25.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:25.905+0000 7f61357fa640 1 -- 172.21.15.43:0/2383892001 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f612005aa80 con 0x7f61380fb420 2024-09-11T08:31:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.083+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f613810bce0 con 0x7f61380fb420 2024-09-11T08:31:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.084+0000 7f61357fa640 1 -- 172.21.15.43:0/2383892001 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f612005a420 con 0x7f61380fb420 2024-09-11T08:31:26.084 INFO:teuthology.orchestra.run.smithi043.stdout:111669149735 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 >> v2:172.21.15.43:6800/588375333 conn(0x7f612406eec0 msgr2=0x7f6124071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 --2- 172.21.15.43:0/2383892001 >> v2:172.21.15.43:6800/588375333 conn(0x7f612406eec0 0x7f6124071370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6128009cf0 tx=0x7f6128035040 comp rx=0 tx=0).stop 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61380fb420 msgr2=0x7f6138113090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61380fb420 0x7f6138113090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f612000b810 tx=0x7f61200091d0 comp rx=0 tx=0).stop 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 shutdown_connections 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 --2- 172.21.15.43:0/2383892001 >> v2:172.21.15.43:6800/588375333 conn(0x7f612406eec0 0x7f6124071370 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61381027f0 0x7f61381135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103190 0x7f613810d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 --2- 172.21.15.43:0/2383892001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61380fb420 0x7f6138113090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 >> 172.21.15.43:0/2383892001 conn(0x7f6138074d00 msgr2=0x7f6138100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:26.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 shutdown_connections 2024-09-11T08:31:26.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:26.086+0000 7f613ebf1640 1 -- 172.21.15.43:0/2383892001 wait complete. 2024-09-11T08:31:26.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:26 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:26.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2383892001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:31:26.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:26 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:26.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2383892001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:31:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:26 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2383892001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:31:26.789 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:26 smithi043 podman[75731]: 2024-09-11 08:31:26.444277252 +0000 UTC m=+0.470909855 container create f826b14b35612e6b5da925665eb911fb133f467ca02385eb41e2e2f7ae557bee (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:27.127 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149729 got 111669149735 for osd.3 2024-09-11T08:31:27.127 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:27.127 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T08:31:27.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:31:27.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-mon[40592]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:27.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:27.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-mon[29355]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:27.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:27.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 podman[75731]: 2024-09-11 08:31:27.225176969 +0000 UTC m=+1.251809574 container init f826b14b35612e6b5da925665eb911fb133f467ca02385eb41e2e2f7ae557bee (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:27.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 podman[75731]: 2024-09-11 08:31:27.230422479 +0000 UTC m=+1.257055083 container start f826b14b35612e6b5da925665eb911fb133f467ca02385eb41e2e2f7ae557bee (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:27.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:27.244Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:31:27.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:27.244Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:31:27.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:27.245Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-09-11T08:31:27.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:27.246Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:31:27.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:27.273Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:31:27.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:27.274Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:31:27.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:27.277Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:31:27.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:27.277Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:31:27.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 bash[75731]: f826b14b35612e6b5da925665eb911fb133f467ca02385eb41e2e2f7ae557bee 2024-09-11T08:31:27.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:27 smithi043 systemd[1]: Started Ceph alertmanager.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:31:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:27 smithi172 ceph-mon[32987]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:27.811 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:31:27] ENGINE Bus STOPPING 2024-09-11T08:31:28.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:31:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:31:28.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:31:27] ENGINE Bus STOPPED 2024-09-11T08:31:28.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:31:27] ENGINE Bus STARTING 2024-09-11T08:31:28.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:31:27] ENGINE Serving on http://:::9283 2024-09-11T08:31:28.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:31:27 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:31:27] ENGINE Bus STARTED 2024-09-11T08:31:28.556 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.556 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.556 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[40592]: Deploying daemon grafana.a on smithi172 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:31:28.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:31:28.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:28 smithi043 ceph-mon[29355]: Deploying daemon grafana.a on smithi172 2024-09-11T08:31:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:31:28.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:31:28.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:28.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:28 smithi172 ceph-mon[32987]: Deploying daemon grafana.a on smithi172 2024-09-11T08:31:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 -- 172.21.15.43:0/3837743099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 msgr2=0x7fb064108510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3837743099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 0x7fb064108510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009cd0 tx=0x7fb05c004d10 comp rx=0 tx=0).stop 2024-09-11T08:31:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 -- 172.21.15.43:0/3837743099 shutdown_connections 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3837743099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 0x7fb064108510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3837743099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb064102090 0x7fb0640fbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3837743099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064101780 0x7fb064101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 -- 172.21.15.43:0/3837743099 >> 172.21.15.43:0/3837743099 conn(0x7fb064074c80 msgr2=0x7fb0640f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 -- 172.21.15.43:0/3837743099 shutdown_connections 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.067+0000 7fb06b07c640 1 -- 172.21.15.43:0/3837743099 wait complete. 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb06b07c640 1 Processor -- start 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb06b07c640 1 -- start start 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb06b07c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 0x7fb0640fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb06b07c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb064101780 0x7fb0640fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb06b07c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064102090 0x7fb0640fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb06b07c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06411aff0 con 0x7fb064102090 2024-09-11T08:31:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb06b07c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06411b160 con 0x7fb064101780 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb06b07c640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06411b320 con 0x7fb0640fc550 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb068e18640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 0x7fb0640fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb069619640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064102090 0x7fb0640fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb068e18640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 0x7fb0640fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:42274/0 (socket says 172.21.15.43:42274) 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb068e18640 1 -- 172.21.15.43:0/3863836356 learned_addr learned my addr 172.21.15.43:0/3863836356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb063fff640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb064101780 0x7fb0640fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb068e18640 1 -- 172.21.15.43:0/3863836356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb064101780 msgr2=0x7fb0640fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb068e18640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb064101780 0x7fb0640fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb068e18640 1 -- 172.21.15.43:0/3863836356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064102090 msgr2=0x7fb0640fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.068+0000 7fb068e18640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064102090 0x7fb0640fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.069+0000 7fb068e18640 1 -- 172.21.15.43:0/3863836356 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c005040 con 0x7fb0640fc550 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.069+0000 7fb069619640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064102090 0x7fb0640fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.069+0000 7fb068e18640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 0x7fb0640fd3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009750 tx=0x7fb04c009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.069+0000 7fb0617fa640 1 -- 172.21.15.43:0/3863836356 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c009990 con 0x7fb0640fc550 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.069+0000 7fb0617fa640 1 -- 172.21.15.43:0/3863836356 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fb04c00f040 con 0x7fb0640fc550 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.069+0000 7fb0617fa640 1 -- 172.21.15.43:0/3863836356 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c013c00 con 0x7fb0640fc550 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.069+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c0098b0 con 0x7fb0640fc550 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.069+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06411b890 con 0x7fb0640fc550 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.070+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0640fd040 con 0x7fb0640fc550 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.071+0000 7fb0617fa640 1 -- 172.21.15.43:0/3863836356 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fb04c013400 con 0x7fb0640fc550 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.071+0000 7fb0617fa640 1 --2- 172.21.15.43:0/3863836356 >> v2:172.21.15.43:6800/588375333 conn(0x7fb05006efb0 0x7fb050071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.071+0000 7fb0617fa640 1 -- 172.21.15.43:0/3863836356 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7fb04c08e3d0 con 0x7fb0640fc550 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.071+0000 7fb063fff640 1 --2- 172.21.15.43:0/3863836356 >> v2:172.21.15.43:6800/588375333 conn(0x7fb05006efb0 0x7fb050071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.072+0000 7fb063fff640 1 --2- 172.21.15.43:0/3863836356 >> v2:172.21.15.43:6800/588375333 conn(0x7fb05006efb0 0x7fb050071460 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb064064fc0 tx=0x7fb05400b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:29.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.075+0000 7fb0617fa640 1 -- 172.21.15.43:0/3863836356 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb04c05b360 con 0x7fb0640fc550 2024-09-11T08:31:29.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.238+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 --> v2:172.21.15.43:6800/588375333 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb06410abb0 con 0x7fb05006efb0 2024-09-11T08:31:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.249+0000 7fb0617fa640 1 -- 172.21.15.43:0/3863836356 <== mgr.14553 v2:172.21.15.43:6800/588375333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384659 (secure 0 0 0) 0x7fb06410abb0 con 0x7fb05006efb0 2024-09-11T08:31:29.253 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:31:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T08:31:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 >> v2:172.21.15.43:6800/588375333 conn(0x7fb05006efb0 msgr2=0x7fb050071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3863836356 >> v2:172.21.15.43:6800/588375333 conn(0x7fb05006efb0 0x7fb050071460 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb064064fc0 tx=0x7fb05400b040 comp rx=0 tx=0).stop 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 msgr2=0x7fb0640fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 0x7fb0640fd3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009750 tx=0x7fb04c009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 shutdown_connections 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3863836356 >> v2:172.21.15.43:6800/588375333 conn(0x7fb05006efb0 0x7fb050071460 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064102090 0x7fb0640fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb064101780 0x7fb0640fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 --2- 172.21.15.43:0/3863836356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0640fc550 0x7fb0640fd3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 >> 172.21.15.43:0/3863836356 conn(0x7fb064074c80 msgr2=0x7fb064071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 shutdown_connections 2024-09-11T08:31:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:29.252+0000 7fb06b07c640 1 -- 172.21.15.43:0/3863836356 wait complete. 2024-09-11T08:31:29.580 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:29 smithi043 ceph-mon[40592]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:29.580 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:29 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:29.246Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000535515s 2024-09-11T08:31:29.580 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:29 smithi043 ceph-mon[29355]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:29 smithi172 ceph-mon[32987]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:29.936 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-11T08:31:28.164197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":931,"num_read_kb":788,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385444,"kb_used_data":6608,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575772,"statfs":{"total":767960285184,"available":765517590528,"internally_reserved":0,"allocated":6766592,"data_stored":3649990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002722"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250295+0000","last_change":"2024-09-11T08:30:02.996268+0000","last_active":"2024-09-11T08:30:14.250295+0000","last_peered":"2024-09-11T08:30:14.250295+0000","last_clean":"2024-09-11T08:30:14.250295+0000","last_became_active":"2024-09-11T08:30:02.995697+0000","last_became_peered":"2024-09-11T08:30:02.995697+0000","last_unstale":"2024-09-11T08:30:14.250295+0000","last_undegraded":"2024-09-11T08:30:14.250295+0000","last_fullsized":"2024-09-11T08:30:14.250295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:55.831568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058698+0000","last_change":"2024-09-11T08:30:05.070463+0000","last_active":"2024-09-11T08:30:14.058698+0000","last_peered":"2024-09-11T08:30:14.058698+0000","last_clean":"2024-09-11T08:30:14.058698+0000","last_became_active":"2024-09-11T08:30:05.070160+0000","last_became_peered":"2024-09-11T08:30:05.070160+0000","last_unstale":"2024-09-11T08:30:14.058698+0000","last_undegraded":"2024-09-11T08:30:14.058698+0000","last_fullsized":"2024-09-11T08:30:14.058698+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:41.965422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.061030+0000","last_change":"2024-09-11T08:30:07.090704+0000","last_active":"2024-09-11T08:30:14.061030+0000","last_peered":"2024-09-11T08:30:14.061030+0000","last_clean":"2024-09-11T08:30:14.061030+0000","last_became_active":"2024-09-11T08:30:07.089958+0000","last_became_peered":"2024-09-11T08:30:07.089958+0000","last_unstale":"2024-09-11T08:30:14.061030+0000","last_undegraded":"2024-09-11T08:30:14.061030+0000","last_fullsized":"2024-09-11T08:30:14.061030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:31.655881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058850+0000","last_change":"2024-09-11T08:30:09.102309+0000","last_active":"2024-09-11T08:30:14.058850+0000","last_peered":"2024-09-11T08:30:14.058850+0000","last_clean":"2024-09-11T08:30:14.058850+0000","last_became_active":"2024-09-11T08:30:09.102007+0000","last_became_peered":"2024-09-11T08:30:09.102007+0000","last_unstale":"2024-09-11T08:30:14.058850+0000","last_undegraded":"2024-09-11T08:30:14.058850+0000","last_fullsized":"2024-09-11T08:30:14.058850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:07:50.975901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678270+0000","last_change":"2024-09-11T08:30:09.643443+0000","last_active":"2024-09-11T08:30:14.678270+0000","last_peered":"2024-09-11T08:30:14.678270+0000","last_clean":"2024-09-11T08:30:14.678270+0000","last_became_active":"2024-09-11T08:30:09.643122+0000","last_became_peered":"2024-09-11T08:30:09.643122+0000","last_unstale":"2024-09-11T08:30:14.678270+0000","last_undegraded":"2024-09-11T08:30:14.678270+0000","last_fullsized":"2024-09-11T08:30:14.678270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:55.728852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679045+0000","last_change":"2024-09-11T08:30:02.986951+0000","last_active":"2024-09-11T08:30:14.679045+0000","last_peered":"2024-09-11T08:30:14.679045+0000","last_clean":"2024-09-11T08:30:14.679045+0000","last_became_active":"2024-09-11T08:30:02.986647+0000","last_became_peered":"2024-09-11T08:30:02.986647+0000","last_unstale":"2024-09-11T08:30:14.679045+0000","last_undegraded":"2024-09-11T08:30:14.679045+0000","last_fullsized":"2024-09-11T08:30:14.679045+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:52.161933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678533+0000","last_change":"2024-09-11T08:30:05.077989+0000","last_active":"2024-09-11T08:30:14.678533+0000","last_peered":"2024-09-11T08:30:14.678533+0000","last_clean":"2024-09-11T08:30:14.678533+0000","last_became_active":"2024-09-11T08:30:05.076716+0000","last_became_peered":"2024-09-11T08:30:05.076716+0000","last_unstale":"2024-09-11T08:30:14.678533+0000","last_undegraded":"2024-09-11T08:30:14.678533+0000","last_fullsized":"2024-09-11T08:30:14.678533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:32.636665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058694+0000","last_change":"2024-09-11T08:30:07.094953+0000","last_active":"2024-09-11T08:30:14.058694+0000","last_peered":"2024-09-11T08:30:14.058694+0000","last_clean":"2024-09-11T08:30:14.058694+0000","last_became_active":"2024-09-11T08:30:07.094482+0000","last_became_peered":"2024-09-11T08:30:07.094482+0000","last_unstale":"2024-09-11T08:30:14.058694+0000","last_undegraded":"2024-09-11T08:30:14.058694+0000","last_fullsized":"2024-09-11T08:30:14.058694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:43:53.526055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679846+0000","last_change":"2024-09-11T08:30:02.991821+0000","last_active":"2024-09-11T08:30:14.679846+0000","last_peered":"2024-09-11T08:30:14.679846+0000","last_clean":"2024-09-11T08:30:14.679846+0000","last_became_active":"2024-09-11T08:30:02.991542+0000","last_became_peered":"2024-09-11T08:30:02.991542+0000","last_unstale":"2024-09-11T08:30:14.679846+0000","last_undegraded":"2024-09-11T08:30:14.679846+0000","last_fullsized":"2024-09-11T08:30:14.679846+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:11.128311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059007+0000","last_change":"2024-09-11T08:30:05.075388+0000","last_active":"2024-09-11T08:30:14.059007+0000","last_peered":"2024-09-11T08:30:14.059007+0000","last_clean":"2024-09-11T08:30:14.059007+0000","last_became_active":"2024-09-11T08:30:05.075107+0000","last_became_peered":"2024-09-11T08:30:05.075107+0000","last_unstale":"2024-09-11T08:30:14.059007+0000","last_undegraded":"2024-09-11T08:30:14.059007+0000","last_fullsized":"2024-09-11T08:30:14.059007+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:43:37.495052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679231+0000","last_change":"2024-09-11T08:30:07.100815+0000","last_active":"2024-09-11T08:30:14.679231+0000","last_peered":"2024-09-11T08:30:14.679231+0000","last_clean":"2024-09-11T08:30:14.679231+0000","last_became_active":"2024-09-11T08:30:07.100134+0000","last_became_peered":"2024-09-11T08:30:07.100134+0000","last_unstale":"2024-09-11T08:30:14.679231+0000","last_undegraded":"2024-09-11T08:30:14.679231+0000","last_fullsized":"2024-09-11T08:30:14.679231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:27.216340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249440+0000","last_change":"2024-09-11T08:30:09.114943+0000","last_active":"2024-09-11T08:30:14.249440+0000","last_peered":"2024-09-11T08:30:14.249440+0000","last_clean":"2024-09-11T08:30:14.249440+0000","last_became_active":"2024-09-11T08:30:09.114511+0000","last_became_peered":"2024-09-11T08:30:09.114511+0000","last_unstale":"2024-09-11T08:30:14.249440+0000","last_undegraded":"2024-09-11T08:30:14.249440+0000","last_fullsized":"2024-09-11T08:30:14.249440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:30:32.217603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679565+0000","last_change":"2024-09-11T08:30:02.994076+0000","last_active":"2024-09-11T08:30:14.679565+0000","last_peered":"2024-09-11T08:30:14.679565+0000","last_clean":"2024-09-11T08:30:14.679565+0000","last_became_active":"2024-09-11T08:30:02.993609+0000","last_became_peered":"2024-09-11T08:30:02.993609+0000","last_unstale":"2024-09-11T08:30:14.679565+0000","last_undegraded":"2024-09-11T08:30:14.679565+0000","last_fullsized":"2024-09-11T08:30:14.679565+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:29.826139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":26,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058989+0000","last_change":"2024-09-11T08:30:05.073968+0000","last_active":"2024-09-11T08:30:14.058989+0000","last_peered":"2024-09-11T08:30:14.058989+0000","last_clean":"2024-09-11T08:30:14.058989+0000","last_became_active":"2024-09-11T08:30:05.073635+0000","last_became_peered":"2024-09-11T08:30:05.073635+0000","last_unstale":"2024-09-11T08:30:14.058989+0000","last_undegraded":"2024-09-11T08:30:14.058989+0000","last_fullsized":"2024-09-11T08:30:14.058989+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:58:48.408406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678500+0000","last_change":"2024-09-11T08:30:07.092049+0000","last_active":"2024-09-11T08:30:14.678500+0000","last_peered":"2024-09-11T08:30:14.678500+0000","last_clean":"2024-09-11T08:30:14.678500+0000","last_became_active":"2024-09-11T08:30:07.091750+0000","last_became_peered":"2024-09-11T08:30:07.091750+0000","last_unstale":"2024-09-11T08:30:14.678500+0000","last_undegraded":"2024-09-11T08:30:14.678500+0000","last_fullsized":"2024-09-11T08:30:14.678500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:20.899947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678422+0000","last_change":"2024-09-11T08:30:09.105281+0000","last_active":"2024-09-11T08:30:14.678422+0000","last_peered":"2024-09-11T08:30:14.678422+0000","last_clean":"2024-09-11T08:30:14.678422+0000","last_became_active":"2024-09-11T08:30:09.104983+0000","last_became_peered":"2024-09-11T08:30:09.104983+0000","last_unstale":"2024-09-11T08:30:14.678422+0000","last_undegraded":"2024-09-11T08:30:14.678422+0000","last_fullsized":"2024-09-11T08:30:14.678422+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:38:35.126318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059323+0000","last_change":"2024-09-11T08:30:09.641951+0000","last_active":"2024-09-11T08:30:14.059323+0000","last_peered":"2024-09-11T08:30:14.059323+0000","last_clean":"2024-09-11T08:30:14.059323+0000","last_became_active":"2024-09-11T08:30:09.640951+0000","last_became_peered":"2024-09-11T08:30:09.640951+0000","last_unstale":"2024-09-11T08:30:14.059323+0000","last_undegraded":"2024-09-11T08:30:14.059323+0000","last_fullsized":"2024-09-11T08:30:14.059323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:51.992594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250303+0000","last_change":"2024-09-11T08:30:02.996395+0000","last_active":"2024-09-11T08:30:14.250303+0000","last_peered":"2024-09-11T08:30:14.250303+0000","last_clean":"2024-09-11T08:30:14.250303+0000","last_became_active":"2024-09-11T08:30:02.996121+0000","last_became_peered":"2024-09-11T08:30:02.996121+0000","last_unstale":"2024-09-11T08:30:14.250303+0000","last_undegraded":"2024-09-11T08:30:14.250303+0000","last_fullsized":"2024-09-11T08:30:14.250303+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:04:57.548246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060170+0000","last_change":"2024-09-11T08:30:05.075898+0000","last_active":"2024-09-11T08:30:14.060170+0000","last_peered":"2024-09-11T08:30:14.060170+0000","last_clean":"2024-09-11T08:30:14.060170+0000","last_became_active":"2024-09-11T08:30:05.075562+0000","last_became_peered":"2024-09-11T08:30:05.075562+0000","last_unstale":"2024-09-11T08:30:14.060170+0000","last_undegraded":"2024-09-11T08:30:14.060170+0000","last_fullsized":"2024-09-11T08:30:14.060170+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:25.617204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060324+0000","last_change":"2024-09-11T08:30:07.101087+0000","last_active":"2024-09-11T08:30:14.060324+0000","last_peered":"2024-09-11T08:30:14.060324+0000","last_clean":"2024-09-11T08:30:14.060324+0000","last_became_active":"2024-09-11T08:30:07.099930+0000","last_became_peered":"2024-09-11T08:30:07.099930+0000","last_unstale":"2024-09-11T08:30:14.060324+0000","last_undegraded":"2024-09-11T08:30:14.060324+0000","last_fullsized":"2024-09-11T08:30:14.060324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:51:07.366130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678960+0000","last_change":"2024-09-11T08:30:09.641223+0000","last_active":"2024-09-11T08:30:14.678960+0000","last_peered":"2024-09-11T08:30:14.678960+0000","last_clean":"2024-09-11T08:30:14.678960+0000","last_became_active":"2024-09-11T08:30:09.640687+0000","last_became_peered":"2024-09-11T08:30:09.640687+0000","last_unstale":"2024-09-11T08:30:14.678960+0000","last_undegraded":"2024-09-11T08:30:14.678960+0000","last_fullsized":"2024-09-11T08:30:14.678960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:20:57.664659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059079+0000","last_change":"2024-09-11T08:30:02.994123+0000","last_active":"2024-09-11T08:30:14.059079+0000","last_peered":"2024-09-11T08:30:14.059079+0000","last_clean":"2024-09-11T08:30:14.059079+0000","last_became_active":"2024-09-11T08:30:02.993832+0000","last_became_peered":"2024-09-11T08:30:02.993832+0000","last_unstale":"2024-09-11T08:30:14.059079+0000","last_undegraded":"2024-09-11T08:30:14.059079+0000","last_fullsized":"2024-09-11T08:30:14.059079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:09.219971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678979+0000","last_change":"2024-09-11T08:30:05.084156+0000","last_active":"2024-09-11T08:30:14.678979+0000","last_peered":"2024-09-11T08:30:14.678979+0000","last_clean":"2024-09-11T08:30:14.678979+0000","last_became_active":"2024-09-11T08:30:05.083863+0000","last_became_peered":"2024-09-11T08:30:05.083863+0000","last_unstale":"2024-09-11T08:30:14.678979+0000","last_undegraded":"2024-09-11T08:30:14.678979+0000","last_fullsized":"2024-09-11T08:30:14.678979+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:56.915638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059079+0000","last_change":"2024-09-11T08:30:07.093294+0000","last_active":"2024-09-11T08:30:14.059079+0000","last_peered":"2024-09-11T08:30:14.059079+0000","last_clean":"2024-09-11T08:30:14.059079+0000","last_became_active":"2024-09-11T08:30:07.092984+0000","last_became_peered":"2024-09-11T08:30:07.092984+0000","last_unstale":"2024-09-11T08:30:14.059079+0000","last_undegraded":"2024-09-11T08:30:14.059079+0000","last_fullsized":"2024-09-11T08:30:14.059079+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:25:42.406618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678920+0000","last_change":"2024-09-11T08:30:09.111012+0000","last_active":"2024-09-11T08:30:14.678920+0000","last_peered":"2024-09-11T08:30:14.678920+0000","last_clean":"2024-09-11T08:30:14.678920+0000","last_became_active":"2024-09-11T08:30:09.110669+0000","last_became_peered":"2024-09-11T08:30:09.110669+0000","last_unstale":"2024-09-11T08:30:14.678920+0000","last_undegraded":"2024-09-11T08:30:14.678920+0000","last_fullsized":"2024-09-11T08:30:14.678920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:54:05.014760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.061119+0000","last_change":"2024-09-11T08:30:02.992025+0000","last_active":"2024-09-11T08:30:14.061119+0000","last_peered":"2024-09-11T08:30:14.061119+0000","last_clean":"2024-09-11T08:30:14.061119+0000","last_became_active":"2024-09-11T08:30:02.991364+0000","last_became_peered":"2024-09-11T08:30:02.991364+0000","last_unstale":"2024-09-11T08:30:14.061119+0000","last_undegraded":"2024-09-11T08:30:14.061119+0000","last_fullsized":"2024-09-11T08:30:14.061119+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:23.825873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250139+0000","last_change":"2024-09-11T08:30:05.973838+0000","last_active":"2024-09-11T08:30:14.250139+0000","last_peered":"2024-09-11T08:30:14.250139+0000","last_clean":"2024-09-11T08:30:14.250139+0000","last_became_active":"2024-09-11T08:30:05.973510+0000","last_became_peered":"2024-09-11T08:30:05.973510+0000","last_unstale":"2024-09-11T08:30:14.250139+0000","last_undegraded":"2024-09-11T08:30:14.250139+0000","last_fullsized":"2024-09-11T08:30:14.250139+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:23.872667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264331+0000","last_change":"2024-09-11T08:30:07.100331+0000","last_active":"2024-09-11T08:31:21.264331+0000","last_peered":"2024-09-11T08:31:21.264331+0000","last_clean":"2024-09-11T08:31:21.264331+0000","last_became_active":"2024-09-11T08:30:07.098073+0000","last_became_peered":"2024-09-11T08:30:07.098073+0000","last_unstale":"2024-09-11T08:31:21.264331+0000","last_undegraded":"2024-09-11T08:31:21.264331+0000","last_fullsized":"2024-09-11T08:31:21.264331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:52.832470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060798+0000","last_change":"2024-09-11T08:30:05.082285+0000","last_active":"2024-09-11T08:30:14.060798+0000","last_peered":"2024-09-11T08:30:14.060798+0000","last_clean":"2024-09-11T08:30:14.060798+0000","last_became_active":"2024-09-11T08:30:05.081895+0000","last_became_peered":"2024-09-11T08:30:05.081895+0000","last_unstale":"2024-09-11T08:30:14.060798+0000","last_undegraded":"2024-09-11T08:30:14.060798+0000","last_fullsized":"2024-09-11T08:30:14.060798+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:41.019175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678588+0000","last_change":"2024-09-11T08:30:02.991948+0000","last_active":"2024-09-11T08:30:14.678588+0000","last_peered":"2024-09-11T08:30:14.678588+0000","last_clean":"2024-09-11T08:30:14.678588+0000","last_became_active":"2024-09-11T08:30:02.991704+0000","last_became_peered":"2024-09-11T08:30:02.991704+0000","last_unstale":"2024-09-11T08:30:14.678588+0000","last_undegraded":"2024-09-11T08:30:14.678588+0000","last_fullsized":"2024-09-11T08:30:14.678588+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:15.657730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264330+0000","last_change":"2024-09-11T08:30:07.098303+0000","last_active":"2024-09-11T08:31:21.264330+0000","last_peered":"2024-09-11T08:31:21.264330+0000","last_clean":"2024-09-11T08:31:21.264330+0000","last_became_active":"2024-09-11T08:30:07.097999+0000","last_became_peered":"2024-09-11T08:30:07.097999+0000","last_unstale":"2024-09-11T08:31:21.264330+0000","last_undegraded":"2024-09-11T08:31:21.264330+0000","last_fullsized":"2024-09-11T08:31:21.264330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:28.511801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679588+0000","last_change":"2024-09-11T08:30:09.112526+0000","last_active":"2024-09-11T08:30:14.679588+0000","last_peered":"2024-09-11T08:30:14.679588+0000","last_clean":"2024-09-11T08:30:14.679588+0000","last_became_active":"2024-09-11T08:30:09.112313+0000","last_became_peered":"2024-09-11T08:30:09.112313+0000","last_unstale":"2024-09-11T08:30:14.679588+0000","last_undegraded":"2024-09-11T08:30:14.679588+0000","last_fullsized":"2024-09-11T08:30:14.679588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:38:41.753218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678254+0000","last_change":"2024-09-11T08:30:05.975282+0000","last_active":"2024-09-11T08:30:14.678254+0000","last_peered":"2024-09-11T08:30:14.678254+0000","last_clean":"2024-09-11T08:30:14.678254+0000","last_became_active":"2024-09-11T08:30:05.974834+0000","last_became_peered":"2024-09-11T08:30:05.974834+0000","last_unstale":"2024-09-11T08:30:14.678254+0000","last_undegraded":"2024-09-11T08:30:14.678254+0000","last_fullsized":"2024-09-11T08:30:14.678254+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:26.693828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679383+0000","last_change":"2024-09-11T08:30:02.997878+0000","last_active":"2024-09-11T08:30:14.679383+0000","last_peered":"2024-09-11T08:30:14.679383+0000","last_clean":"2024-09-11T08:30:14.679383+0000","last_became_active":"2024-09-11T08:30:02.997570+0000","last_became_peered":"2024-09-11T08:30:02.997570+0000","last_unstale":"2024-09-11T08:30:14.679383+0000","last_undegraded":"2024-09-11T08:30:14.679383+0000","last_fullsized":"2024-09-11T08:30:14.679383+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:43:07.844508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058712+0000","last_change":"2024-09-11T08:30:07.095053+0000","last_active":"2024-09-11T08:30:14.058712+0000","last_peered":"2024-09-11T08:30:14.058712+0000","last_clean":"2024-09-11T08:30:14.058712+0000","last_became_active":"2024-09-11T08:30:07.093412+0000","last_became_peered":"2024-09-11T08:30:07.093412+0000","last_unstale":"2024-09-11T08:30:14.058712+0000","last_undegraded":"2024-09-11T08:30:14.058712+0000","last_fullsized":"2024-09-11T08:30:14.058712+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:19.379515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058934+0000","last_change":"2024-09-11T08:30:09.108342+0000","last_active":"2024-09-11T08:30:14.058934+0000","last_peered":"2024-09-11T08:30:14.058934+0000","last_clean":"2024-09-11T08:30:14.058934+0000","last_became_active":"2024-09-11T08:30:09.108078+0000","last_became_peered":"2024-09-11T08:30:09.108078+0000","last_unstale":"2024-09-11T08:30:14.058934+0000","last_undegraded":"2024-09-11T08:30:14.058934+0000","last_fullsized":"2024-09-11T08:30:14.058934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:51:15.578994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058765+0000","last_change":"2024-09-11T08:30:05.068876+0000","last_active":"2024-09-11T08:30:14.058765+0000","last_peered":"2024-09-11T08:30:14.058765+0000","last_clean":"2024-09-11T08:30:14.058765+0000","last_became_active":"2024-09-11T08:30:05.068383+0000","last_became_peered":"2024-09-11T08:30:05.068383+0000","last_unstale":"2024-09-11T08:30:14.058765+0000","last_undegraded":"2024-09-11T08:30:14.058765+0000","last_fullsized":"2024-09-11T08:30:14.058765+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:04.854359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250331+0000","last_change":"2024-09-11T08:30:02.995890+0000","last_active":"2024-09-11T08:30:14.250331+0000","last_peered":"2024-09-11T08:30:14.250331+0000","last_clean":"2024-09-11T08:30:14.250331+0000","last_became_active":"2024-09-11T08:30:02.995557+0000","last_became_peered":"2024-09-11T08:30:02.995557+0000","last_unstale":"2024-09-11T08:30:14.250331+0000","last_undegraded":"2024-09-11T08:30:14.250331+0000","last_fullsized":"2024-09-11T08:30:14.250331+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:03.407212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264275+0000","last_change":"2024-09-11T08:30:07.094535+0000","last_active":"2024-09-11T08:31:21.264275+0000","last_peered":"2024-09-11T08:31:21.264275+0000","last_clean":"2024-09-11T08:31:21.264275+0000","last_became_active":"2024-09-11T08:30:07.094220+0000","last_became_peered":"2024-09-11T08:30:07.094220+0000","last_unstale":"2024-09-11T08:31:21.264275+0000","last_undegraded":"2024-09-11T08:31:21.264275+0000","last_fullsized":"2024-09-11T08:31:21.264275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:12:09.986404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678477+0000","last_change":"2024-09-11T08:30:09.643340+0000","last_active":"2024-09-11T08:30:14.678477+0000","last_peered":"2024-09-11T08:30:14.678477+0000","last_clean":"2024-09-11T08:30:14.678477+0000","last_became_active":"2024-09-11T08:30:09.643060+0000","last_became_peered":"2024-09-11T08:30:09.643060+0000","last_unstale":"2024-09-11T08:30:14.678477+0000","last_undegraded":"2024-09-11T08:30:14.678477+0000","last_fullsized":"2024-09-11T08:30:14.678477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:13.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059928+0000","last_change":"2024-09-11T08:30:05.073081+0000","last_active":"2024-09-11T08:30:14.059928+0000","last_peered":"2024-09-11T08:30:14.059928+0000","last_clean":"2024-09-11T08:30:14.059928+0000","last_became_active":"2024-09-11T08:30:05.072793+0000","last_became_peered":"2024-09-11T08:30:05.072793+0000","last_unstale":"2024-09-11T08:30:14.059928+0000","last_undegraded":"2024-09-11T08:30:14.059928+0000","last_fullsized":"2024-09-11T08:30:14.059928+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:07:45.710385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679896+0000","last_change":"2024-09-11T08:30:02.991940+0000","last_active":"2024-09-11T08:30:14.679896+0000","last_peered":"2024-09-11T08:30:14.679896+0000","last_clean":"2024-09-11T08:30:14.679896+0000","last_became_active":"2024-09-11T08:30:02.991660+0000","last_became_peered":"2024-09-11T08:30:02.991660+0000","last_unstale":"2024-09-11T08:30:14.679896+0000","last_undegraded":"2024-09-11T08:30:14.679896+0000","last_fullsized":"2024-09-11T08:30:14.679896+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:50.015166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264164+0000","last_change":"2024-09-11T08:30:07.103729+0000","last_active":"2024-09-11T08:31:21.264164+0000","last_peered":"2024-09-11T08:31:21.264164+0000","last_clean":"2024-09-11T08:31:21.264164+0000","last_became_active":"2024-09-11T08:30:07.103465+0000","last_became_peered":"2024-09-11T08:30:07.103465+0000","last_unstale":"2024-09-11T08:31:21.264164+0000","last_undegraded":"2024-09-11T08:31:21.264164+0000","last_fullsized":"2024-09-11T08:31:21.264164+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:23.248934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249590+0000","last_change":"2024-09-11T08:30:09.112918+0000","last_active":"2024-09-11T08:30:14.249590+0000","last_peered":"2024-09-11T08:30:14.249590+0000","last_clean":"2024-09-11T08:30:14.249590+0000","last_became_active":"2024-09-11T08:30:09.112569+0000","last_became_peered":"2024-09-11T08:30:09.112569+0000","last_unstale":"2024-09-11T08:30:14.249590+0000","last_undegraded":"2024-09-11T08:30:14.249590+0000","last_fullsized":"2024-09-11T08:30:14.249590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:43:44.615629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250102+0000","last_change":"2024-09-11T08:30:05.973857+0000","last_active":"2024-09-11T08:30:14.250102+0000","last_peered":"2024-09-11T08:30:14.250102+0000","last_clean":"2024-09-11T08:30:14.250102+0000","last_became_active":"2024-09-11T08:30:05.973576+0000","last_became_peered":"2024-09-11T08:30:05.973576+0000","last_unstale":"2024-09-11T08:30:14.250102+0000","last_undegraded":"2024-09-11T08:30:14.250102+0000","last_fullsized":"2024-09-11T08:30:14.250102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:56:58.275016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.061304+0000","last_change":"2024-09-11T08:30:02.990920+0000","last_active":"2024-09-11T08:30:14.061304+0000","last_peered":"2024-09-11T08:30:14.061304+0000","last_clean":"2024-09-11T08:30:14.061304+0000","last_became_active":"2024-09-11T08:30:02.990645+0000","last_became_peered":"2024-09-11T08:30:02.990645+0000","last_unstale":"2024-09-11T08:30:14.061304+0000","last_undegraded":"2024-09-11T08:30:14.061304+0000","last_fullsized":"2024-09-11T08:30:14.061304+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:57:53.449269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678235+0000","last_change":"2024-09-11T08:30:07.102639+0000","last_active":"2024-09-11T08:30:14.678235+0000","last_peered":"2024-09-11T08:30:14.678235+0000","last_clean":"2024-09-11T08:30:14.678235+0000","last_became_active":"2024-09-11T08:30:07.102326+0000","last_became_peered":"2024-09-11T08:30:07.102326+0000","last_unstale":"2024-09-11T08:30:14.678235+0000","last_undegraded":"2024-09-11T08:30:14.678235+0000","last_fullsized":"2024-09-11T08:30:14.678235+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:50:58.747353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059634+0000","last_change":"2024-09-11T08:30:09.641083+0000","last_active":"2024-09-11T08:30:14.059634+0000","last_peered":"2024-09-11T08:30:14.059634+0000","last_clean":"2024-09-11T08:30:14.059634+0000","last_became_active":"2024-09-11T08:30:09.640758+0000","last_became_peered":"2024-09-11T08:30:09.640758+0000","last_unstale":"2024-09-11T08:30:14.059634+0000","last_undegraded":"2024-09-11T08:30:14.059634+0000","last_fullsized":"2024-09-11T08:30:14.059634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:33.363406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":81,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264156+0000","last_change":"2024-09-11T08:30:05.082192+0000","last_active":"2024-09-11T08:31:21.264156+0000","last_peered":"2024-09-11T08:31:21.264156+0000","last_clean":"2024-09-11T08:31:21.264156+0000","last_became_active":"2024-09-11T08:30:05.081856+0000","last_became_peered":"2024-09-11T08:30:05.081856+0000","last_unstale":"2024-09-11T08:31:21.264156+0000","last_undegraded":"2024-09-11T08:31:21.264156+0000","last_fullsized":"2024-09-11T08:31:21.264156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:48.332041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059170+0000","last_change":"2024-09-11T08:30:02.997130+0000","last_active":"2024-09-11T08:30:14.059170+0000","last_peered":"2024-09-11T08:30:14.059170+0000","last_clean":"2024-09-11T08:30:14.059170+0000","last_became_active":"2024-09-11T08:30:02.996767+0000","last_became_peered":"2024-09-11T08:30:02.996767+0000","last_unstale":"2024-09-11T08:30:14.059170+0000","last_undegraded":"2024-09-11T08:30:14.059170+0000","last_fullsized":"2024-09-11T08:30:14.059170+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:00.156489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"57'2","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678989+0000","last_change":"2024-09-11T08:30:07.083427+0000","last_active":"2024-09-11T08:30:14.678989+0000","last_peered":"2024-09-11T08:30:14.678989+0000","last_clean":"2024-09-11T08:30:14.678989+0000","last_became_active":"2024-09-11T08:30:02.993706+0000","last_became_peered":"2024-09-11T08:30:02.993706+0000","last_unstale":"2024-09-11T08:30:14.678989+0000","last_undegraded":"2024-09-11T08:30:14.678989+0000","last_fullsized":"2024-09-11T08:30:14.678989+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:56:09.045984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038253099999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250068+0000","last_change":"2024-09-11T08:30:07.098131+0000","last_active":"2024-09-11T08:30:14.250068+0000","last_peered":"2024-09-11T08:30:14.250068+0000","last_clean":"2024-09-11T08:30:14.250068+0000","last_became_active":"2024-09-11T08:30:07.097851+0000","last_became_peered":"2024-09-11T08:30:07.097851+0000","last_unstale":"2024-09-11T08:30:14.250068+0000","last_undegraded":"2024-09-11T08:30:14.250068+0000","last_fullsized":"2024-09-11T08:30:14.250068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:55.270252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678593+0000","last_change":"2024-09-11T08:30:09.108749+0000","last_active":"2024-09-11T08:30:14.678593+0000","last_peered":"2024-09-11T08:30:14.678593+0000","last_clean":"2024-09-11T08:30:14.678593+0000","last_became_active":"2024-09-11T08:30:09.108464+0000","last_became_peered":"2024-09-11T08:30:09.108464+0000","last_unstale":"2024-09-11T08:30:14.678593+0000","last_undegraded":"2024-09-11T08:30:14.678593+0000","last_fullsized":"2024-09-11T08:30:14.678593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:13.637671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060884+0000","last_change":"2024-09-11T08:30:05.076405+0000","last_active":"2024-09-11T08:30:14.060884+0000","last_peered":"2024-09-11T08:30:14.060884+0000","last_clean":"2024-09-11T08:30:14.060884+0000","last_became_active":"2024-09-11T08:30:05.071843+0000","last_became_peered":"2024-09-11T08:30:05.071843+0000","last_unstale":"2024-09-11T08:30:14.060884+0000","last_undegraded":"2024-09-11T08:30:14.060884+0000","last_fullsized":"2024-09-11T08:30:14.060884+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:22.989769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060280+0000","last_change":"2024-09-11T08:30:02.985719+0000","last_active":"2024-09-11T08:30:14.060280+0000","last_peered":"2024-09-11T08:30:14.060280+0000","last_clean":"2024-09-11T08:30:14.060280+0000","last_became_active":"2024-09-11T08:30:02.985411+0000","last_became_peered":"2024-09-11T08:30:02.985411+0000","last_unstale":"2024-09-11T08:30:14.060280+0000","last_undegraded":"2024-09-11T08:30:14.060280+0000","last_fullsized":"2024-09-11T08:30:14.060280+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:32.481440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"55'1","reported_seq":24,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059093+0000","last_change":"2024-09-11T08:30:07.079602+0000","last_active":"2024-09-11T08:30:14.059093+0000","last_peered":"2024-09-11T08:30:14.059093+0000","last_clean":"2024-09-11T08:30:14.059093+0000","last_became_active":"2024-09-11T08:30:02.990758+0000","last_became_peered":"2024-09-11T08:30:02.990758+0000","last_unstale":"2024-09-11T08:30:14.059093+0000","last_undegraded":"2024-09-11T08:30:14.059093+0000","last_fullsized":"2024-09-11T08:30:14.059093+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:37.284174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000370121,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059913+0000","last_change":"2024-09-11T08:30:07.092239+0000","last_active":"2024-09-11T08:30:14.059913+0000","last_peered":"2024-09-11T08:30:14.059913+0000","last_clean":"2024-09-11T08:30:14.059913+0000","last_became_active":"2024-09-11T08:30:07.091956+0000","last_became_peered":"2024-09-11T08:30:07.091956+0000","last_unstale":"2024-09-11T08:30:14.059913+0000","last_undegraded":"2024-09-11T08:30:14.059913+0000","last_fullsized":"2024-09-11T08:30:14.059913+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:08:07.546081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678788+0000","last_change":"2024-09-11T08:30:09.643663+0000","last_active":"2024-09-11T08:30:14.678788+0000","last_peered":"2024-09-11T08:30:14.678788+0000","last_clean":"2024-09-11T08:30:14.678788+0000","last_became_active":"2024-09-11T08:30:09.643136+0000","last_became_peered":"2024-09-11T08:30:09.643136+0000","last_unstale":"2024-09-11T08:30:14.678788+0000","last_undegraded":"2024-09-11T08:30:14.678788+0000","last_fullsized":"2024-09-11T08:30:14.678788+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:18.580621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":34,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250222+0000","last_change":"2024-09-11T08:30:05.079482+0000","last_active":"2024-09-11T08:30:14.250222+0000","last_peered":"2024-09-11T08:30:14.250222+0000","last_clean":"2024-09-11T08:30:14.250222+0000","last_became_active":"2024-09-11T08:30:05.079159+0000","last_became_peered":"2024-09-11T08:30:05.079159+0000","last_unstale":"2024-09-11T08:30:14.250222+0000","last_undegraded":"2024-09-11T08:30:14.250222+0000","last_fullsized":"2024-09-11T08:30:14.250222+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:32:37.893619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249687+0000","last_change":"2024-09-11T08:30:02.995420+0000","last_active":"2024-09-11T08:30:14.249687+0000","last_peered":"2024-09-11T08:30:14.249687+0000","last_clean":"2024-09-11T08:30:14.249687+0000","last_became_active":"2024-09-11T08:30:02.995197+0000","last_became_peered":"2024-09-11T08:30:02.995197+0000","last_unstale":"2024-09-11T08:30:14.249687+0000","last_undegraded":"2024-09-11T08:30:14.249687+0000","last_fullsized":"2024-09-11T08:30:14.249687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:10.531670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'5","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:26.477166+0000","last_change":"2024-09-11T08:30:07.088060+0000","last_active":"2024-09-11T08:31:26.477166+0000","last_peered":"2024-09-11T08:31:26.477166+0000","last_clean":"2024-09-11T08:31:26.477166+0000","last_became_active":"2024-09-11T08:30:02.997083+0000","last_became_peered":"2024-09-11T08:30:02.997083+0000","last_unstale":"2024-09-11T08:31:26.477166+0000","last_undegraded":"2024-09-11T08:31:26.477166+0000","last_fullsized":"2024-09-11T08:31:26.477166+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:36.015109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039662399999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.680068+0000","last_change":"2024-09-11T08:30:07.096694+0000","last_active":"2024-09-11T08:30:14.680068+0000","last_peered":"2024-09-11T08:30:14.680068+0000","last_clean":"2024-09-11T08:30:14.680068+0000","last_became_active":"2024-09-11T08:30:07.096415+0000","last_became_peered":"2024-09-11T08:30:07.096415+0000","last_unstale":"2024-09-11T08:30:14.680068+0000","last_undegraded":"2024-09-11T08:30:14.680068+0000","last_fullsized":"2024-09-11T08:30:14.680068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:52.859229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060158+0000","last_change":"2024-09-11T08:30:09.103602+0000","last_active":"2024-09-11T08:30:14.060158+0000","last_peered":"2024-09-11T08:30:14.060158+0000","last_clean":"2024-09-11T08:30:14.060158+0000","last_became_active":"2024-09-11T08:30:09.103343+0000","last_became_peered":"2024-09-11T08:30:09.103343+0000","last_unstale":"2024-09-11T08:30:14.060158+0000","last_undegraded":"2024-09-11T08:30:14.060158+0000","last_fullsized":"2024-09-11T08:30:14.060158+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:55.546078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679330+0000","last_change":"2024-09-11T08:30:05.974728+0000","last_active":"2024-09-11T08:30:14.679330+0000","last_peered":"2024-09-11T08:30:14.679330+0000","last_clean":"2024-09-11T08:30:14.679330+0000","last_became_active":"2024-09-11T08:30:05.974443+0000","last_became_peered":"2024-09-11T08:30:05.974443+0000","last_unstale":"2024-09-11T08:30:14.679330+0000","last_undegraded":"2024-09-11T08:30:14.679330+0000","last_fullsized":"2024-09-11T08:30:14.679330+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:28:16.980240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678633+0000","last_change":"2024-09-11T08:30:02.990923+0000","last_active":"2024-09-11T08:30:14.678633+0000","last_peered":"2024-09-11T08:30:14.678633+0000","last_clean":"2024-09-11T08:30:14.678633+0000","last_became_active":"2024-09-11T08:30:02.990591+0000","last_became_peered":"2024-09-11T08:30:02.990591+0000","last_unstale":"2024-09-11T08:30:14.678633+0000","last_undegraded":"2024-09-11T08:30:14.678633+0000","last_fullsized":"2024-09-11T08:30:14.678633+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:54:31.959997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"62'79","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:30:44.206326+0000","last_change":"2024-09-11T08:29:50.832904+0000","last_active":"2024-09-11T08:30:44.206326+0000","last_peered":"2024-09-11T08:30:44.206326+0000","last_clean":"2024-09-11T08:30:44.206326+0000","last_became_active":"2024-09-11T08:29:50.828087+0000","last_became_peered":"2024-09-11T08:29:50.828087+0000","last_unstale":"2024-09-11T08:30:44.206326+0000","last_undegraded":"2024-09-11T08:30:44.206326+0000","last_fullsized":"2024-09-11T08:30:44.206326+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:27:53.850268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:27:53.850268+0000","last_clean_scrub_stamp":"2024-09-11T08:27:53.850268+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:48.200685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679048+0000","last_change":"2024-09-11T08:30:07.093476+0000","last_active":"2024-09-11T08:30:14.679048+0000","last_peered":"2024-09-11T08:30:14.679048+0000","last_clean":"2024-09-11T08:30:14.679048+0000","last_became_active":"2024-09-11T08:30:07.092977+0000","last_became_peered":"2024-09-11T08:30:07.092977+0000","last_unstale":"2024-09-11T08:30:14.679048+0000","last_undegraded":"2024-09-11T08:30:14.679048+0000","last_fullsized":"2024-09-11T08:30:14.679048+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:26:11.881790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679987+0000","last_change":"2024-09-11T08:30:09.105320+0000","last_active":"2024-09-11T08:30:14.679987+0000","last_peered":"2024-09-11T08:30:14.679987+0000","last_clean":"2024-09-11T08:30:14.679987+0000","last_became_active":"2024-09-11T08:30:09.105042+0000","last_became_peered":"2024-09-11T08:30:09.105042+0000","last_unstale":"2024-09-11T08:30:14.679987+0000","last_undegraded":"2024-09-11T08:30:14.679987+0000","last_fullsized":"2024-09-11T08:30:14.679987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:45.609579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059502+0000","last_change":"2024-09-11T08:30:05.070700+0000","last_active":"2024-09-11T08:30:14.059502+0000","last_peered":"2024-09-11T08:30:14.059502+0000","last_clean":"2024-09-11T08:30:14.059502+0000","last_became_active":"2024-09-11T08:30:05.070351+0000","last_became_peered":"2024-09-11T08:30:05.070351+0000","last_unstale":"2024-09-11T08:30:14.059502+0000","last_undegraded":"2024-09-11T08:30:14.059502+0000","last_fullsized":"2024-09-11T08:30:14.059502+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:22:42.588741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058957+0000","last_change":"2024-09-11T08:30:02.995286+0000","last_active":"2024-09-11T08:30:14.058957+0000","last_peered":"2024-09-11T08:30:14.058957+0000","last_clean":"2024-09-11T08:30:14.058957+0000","last_became_active":"2024-09-11T08:30:02.994970+0000","last_became_peered":"2024-09-11T08:30:02.994970+0000","last_unstale":"2024-09-11T08:30:14.058957+0000","last_undegraded":"2024-09-11T08:30:14.058957+0000","last_fullsized":"2024-09-11T08:30:14.058957+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:30.226149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679957+0000","last_change":"2024-09-11T08:30:07.096814+0000","last_active":"2024-09-11T08:30:14.679957+0000","last_peered":"2024-09-11T08:30:14.679957+0000","last_clean":"2024-09-11T08:30:14.679957+0000","last_became_active":"2024-09-11T08:30:07.096536+0000","last_became_peered":"2024-09-11T08:30:07.096536+0000","last_unstale":"2024-09-11T08:30:14.679957+0000","last_undegraded":"2024-09-11T08:30:14.679957+0000","last_fullsized":"2024-09-11T08:30:14.679957+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:26:17.734806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678825+0000","last_change":"2024-09-11T08:30:09.641050+0000","last_active":"2024-09-11T08:30:14.678825+0000","last_peered":"2024-09-11T08:30:14.678825+0000","last_clean":"2024-09-11T08:30:14.678825+0000","last_became_active":"2024-09-11T08:30:09.640647+0000","last_became_peered":"2024-09-11T08:30:09.640647+0000","last_unstale":"2024-09-11T08:30:14.678825+0000","last_undegraded":"2024-09-11T08:30:14.678825+0000","last_fullsized":"2024-09-11T08:30:14.678825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:30.559696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059412+0000","last_change":"2024-09-11T08:30:05.080399+0000","last_active":"2024-09-11T08:30:14.059412+0000","last_peered":"2024-09-11T08:30:14.059412+0000","last_clean":"2024-09-11T08:30:14.059412+0000","last_became_active":"2024-09-11T08:30:05.079961+0000","last_became_peered":"2024-09-11T08:30:05.079961+0000","last_unstale":"2024-09-11T08:30:14.059412+0000","last_undegraded":"2024-09-11T08:30:14.059412+0000","last_fullsized":"2024-09-11T08:30:14.059412+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:16.095070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678978+0000","last_change":"2024-09-11T08:30:02.992376+0000","last_active":"2024-09-11T08:30:14.678978+0000","last_peered":"2024-09-11T08:30:14.678978+0000","last_clean":"2024-09-11T08:30:14.678978+0000","last_became_active":"2024-09-11T08:30:02.992123+0000","last_became_peered":"2024-09-11T08:30:02.992123+0000","last_unstale":"2024-09-11T08:30:14.678978+0000","last_undegraded":"2024-09-11T08:30:14.678978+0000","last_fullsized":"2024-09-11T08:30:14.678978+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:44:08.843691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060243+0000","last_change":"2024-09-11T08:30:07.095086+0000","last_active":"2024-09-11T08:30:14.060243+0000","last_peered":"2024-09-11T08:30:14.060243+0000","last_clean":"2024-09-11T08:30:14.060243+0000","last_became_active":"2024-09-11T08:30:07.094509+0000","last_became_peered":"2024-09-11T08:30:07.094509+0000","last_unstale":"2024-09-11T08:30:14.060243+0000","last_undegraded":"2024-09-11T08:30:14.060243+0000","last_fullsized":"2024-09-11T08:30:14.060243+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:54:50.308666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060085+0000","last_change":"2024-09-11T08:30:09.109370+0000","last_active":"2024-09-11T08:30:14.060085+0000","last_peered":"2024-09-11T08:30:14.060085+0000","last_clean":"2024-09-11T08:30:14.060085+0000","last_became_active":"2024-09-11T08:30:09.109062+0000","last_became_peered":"2024-09-11T08:30:09.109062+0000","last_unstale":"2024-09-11T08:30:14.060085+0000","last_undegraded":"2024-09-11T08:30:14.060085+0000","last_fullsized":"2024-09-11T08:30:14.060085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:39.560907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059180+0000","last_change":"2024-09-11T08:30:05.073973+0000","last_active":"2024-09-11T08:30:14.059180+0000","last_peered":"2024-09-11T08:30:14.059180+0000","last_clean":"2024-09-11T08:30:14.059180+0000","last_became_active":"2024-09-11T08:30:05.073629+0000","last_became_peered":"2024-09-11T08:30:05.073629+0000","last_unstale":"2024-09-11T08:30:14.059180+0000","last_undegraded":"2024-09-11T08:30:14.059180+0000","last_fullsized":"2024-09-11T08:30:14.059180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:12.105558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678921+0000","last_change":"2024-09-11T08:30:02.991778+0000","last_active":"2024-09-11T08:30:14.678921+0000","last_peered":"2024-09-11T08:30:14.678921+0000","last_clean":"2024-09-11T08:30:14.678921+0000","last_became_active":"2024-09-11T08:30:02.991488+0000","last_became_peered":"2024-09-11T08:30:02.991488+0000","last_unstale":"2024-09-11T08:30:14.678921+0000","last_undegraded":"2024-09-11T08:30:14.678921+0000","last_fullsized":"2024-09-11T08:30:14.678921+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:52.278404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264156+0000","last_change":"2024-09-11T08:30:07.090459+0000","last_active":"2024-09-11T08:31:21.264156+0000","last_peered":"2024-09-11T08:31:21.264156+0000","last_clean":"2024-09-11T08:31:21.264156+0000","last_became_active":"2024-09-11T08:30:07.089969+0000","last_became_peered":"2024-09-11T08:30:07.089969+0000","last_unstale":"2024-09-11T08:31:21.264156+0000","last_undegraded":"2024-09-11T08:31:21.264156+0000","last_fullsized":"2024-09-11T08:31:21.264156+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:12:26.011068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059411+0000","last_change":"2024-09-11T08:30:09.109263+0000","last_active":"2024-09-11T08:30:14.059411+0000","last_peered":"2024-09-11T08:30:14.059411+0000","last_clean":"2024-09-11T08:30:14.059411+0000","last_became_active":"2024-09-11T08:30:09.108796+0000","last_became_peered":"2024-09-11T08:30:09.108796+0000","last_unstale":"2024-09-11T08:30:14.059411+0000","last_undegraded":"2024-09-11T08:30:14.059411+0000","last_fullsized":"2024-09-11T08:30:14.059411+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:13.881316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249950+0000","last_change":"2024-09-11T08:30:05.079449+0000","last_active":"2024-09-11T08:30:14.249950+0000","last_peered":"2024-09-11T08:30:14.249950+0000","last_clean":"2024-09-11T08:30:14.249950+0000","last_became_active":"2024-09-11T08:30:05.079106+0000","last_became_peered":"2024-09-11T08:30:05.079106+0000","last_unstale":"2024-09-11T08:30:14.249950+0000","last_undegraded":"2024-09-11T08:30:14.249950+0000","last_fullsized":"2024-09-11T08:30:14.249950+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:26.510253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679614+0000","last_change":"2024-09-11T08:30:02.992069+0000","last_active":"2024-09-11T08:30:14.679614+0000","last_peered":"2024-09-11T08:30:14.679614+0000","last_clean":"2024-09-11T08:30:14.679614+0000","last_became_active":"2024-09-11T08:30:02.991826+0000","last_became_peered":"2024-09-11T08:30:02.991826+0000","last_unstale":"2024-09-11T08:30:14.679614+0000","last_undegraded":"2024-09-11T08:30:14.679614+0000","last_fullsized":"2024-09-11T08:30:14.679614+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:45.876244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059050+0000","last_change":"2024-09-11T08:30:07.093358+0000","last_active":"2024-09-11T08:30:14.059050+0000","last_peered":"2024-09-11T08:30:14.059050+0000","last_clean":"2024-09-11T08:30:14.059050+0000","last_became_active":"2024-09-11T08:30:07.093088+0000","last_became_peered":"2024-09-11T08:30:07.093088+0000","last_unstale":"2024-09-11T08:30:14.059050+0000","last_undegraded":"2024-09-11T08:30:14.059050+0000","last_fullsized":"2024-09-11T08:30:14.059050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:43.377441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249669+0000","last_change":"2024-09-11T08:30:09.115039+0000","last_active":"2024-09-11T08:30:14.249669+0000","last_peered":"2024-09-11T08:30:14.249669+0000","last_clean":"2024-09-11T08:30:14.249669+0000","last_became_active":"2024-09-11T08:30:09.114745+0000","last_became_peered":"2024-09-11T08:30:09.114745+0000","last_unstale":"2024-09-11T08:30:14.249669+0000","last_undegraded":"2024-09-11T08:30:14.249669+0000","last_fullsized":"2024-09-11T08:30:14.249669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:54:08.962931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678793+0000","last_change":"2024-09-11T08:30:05.974089+0000","last_active":"2024-09-11T08:30:14.678793+0000","last_peered":"2024-09-11T08:30:14.678793+0000","last_clean":"2024-09-11T08:30:14.678793+0000","last_became_active":"2024-09-11T08:30:05.973721+0000","last_became_peered":"2024-09-11T08:30:05.973721+0000","last_unstale":"2024-09-11T08:30:14.678793+0000","last_undegraded":"2024-09-11T08:30:14.678793+0000","last_fullsized":"2024-09-11T08:30:14.678793+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:03.167397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679104+0000","last_change":"2024-09-11T08:30:02.994171+0000","last_active":"2024-09-11T08:30:14.679104+0000","last_peered":"2024-09-11T08:30:14.679104+0000","last_clean":"2024-09-11T08:30:14.679104+0000","last_became_active":"2024-09-11T08:30:02.993912+0000","last_became_peered":"2024-09-11T08:30:02.993912+0000","last_unstale":"2024-09-11T08:30:14.679104+0000","last_undegraded":"2024-09-11T08:30:14.679104+0000","last_fullsized":"2024-09-11T08:30:14.679104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:33.612107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059354+0000","last_change":"2024-09-11T08:30:07.099628+0000","last_active":"2024-09-11T08:30:14.059354+0000","last_peered":"2024-09-11T08:30:14.059354+0000","last_clean":"2024-09-11T08:30:14.059354+0000","last_became_active":"2024-09-11T08:30:07.099281+0000","last_became_peered":"2024-09-11T08:30:07.099281+0000","last_unstale":"2024-09-11T08:30:14.059354+0000","last_undegraded":"2024-09-11T08:30:14.059354+0000","last_fullsized":"2024-09-11T08:30:14.059354+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:24.194742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678216+0000","last_change":"2024-09-11T08:30:09.111403+0000","last_active":"2024-09-11T08:30:14.678216+0000","last_peered":"2024-09-11T08:30:14.678216+0000","last_clean":"2024-09-11T08:30:14.678216+0000","last_became_active":"2024-09-11T08:30:09.111094+0000","last_became_peered":"2024-09-11T08:30:09.111094+0000","last_unstale":"2024-09-11T08:30:14.678216+0000","last_undegraded":"2024-09-11T08:30:14.678216+0000","last_fullsized":"2024-09-11T08:30:14.678216+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:40.889055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058869+0000","last_change":"2024-09-11T08:30:05.075489+0000","last_active":"2024-09-11T08:30:14.058869+0000","last_peered":"2024-09-11T08:30:14.058869+0000","last_clean":"2024-09-11T08:30:14.058869+0000","last_became_active":"2024-09-11T08:30:05.075180+0000","last_became_peered":"2024-09-11T08:30:05.075180+0000","last_unstale":"2024-09-11T08:30:14.058869+0000","last_undegraded":"2024-09-11T08:30:14.058869+0000","last_fullsized":"2024-09-11T08:30:14.058869+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:35.928928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.680366+0000","last_change":"2024-09-11T08:30:02.998911+0000","last_active":"2024-09-11T08:30:14.680366+0000","last_peered":"2024-09-11T08:30:14.680366+0000","last_clean":"2024-09-11T08:30:14.680366+0000","last_became_active":"2024-09-11T08:30:02.998485+0000","last_became_peered":"2024-09-11T08:30:02.998485+0000","last_unstale":"2024-09-11T08:30:14.680366+0000","last_undegraded":"2024-09-11T08:30:14.680366+0000","last_fullsized":"2024-09-11T08:30:14.680366+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:53.343966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058875+0000","last_change":"2024-09-11T08:30:07.100401+0000","last_active":"2024-09-11T08:30:14.058875+0000","last_peered":"2024-09-11T08:30:14.058875+0000","last_clean":"2024-09-11T08:30:14.058875+0000","last_became_active":"2024-09-11T08:30:07.099908+0000","last_became_peered":"2024-09-11T08:30:07.099908+0000","last_unstale":"2024-09-11T08:30:14.058875+0000","last_undegraded":"2024-09-11T08:30:14.058875+0000","last_fullsized":"2024-09-11T08:30:14.058875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:44:36.931005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678585+0000","last_change":"2024-09-11T08:30:09.108063+0000","last_active":"2024-09-11T08:30:14.678585+0000","last_peered":"2024-09-11T08:30:14.678585+0000","last_clean":"2024-09-11T08:30:14.678585+0000","last_became_active":"2024-09-11T08:30:09.107722+0000","last_became_peered":"2024-09-11T08:30:09.107722+0000","last_unstale":"2024-09-11T08:30:14.678585+0000","last_undegraded":"2024-09-11T08:30:14.678585+0000","last_fullsized":"2024-09-11T08:30:14.678585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:35.304732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679425+0000","last_change":"2024-09-11T08:30:05.974622+0000","last_active":"2024-09-11T08:30:14.679425+0000","last_peered":"2024-09-11T08:30:14.679425+0000","last_clean":"2024-09-11T08:30:14.679425+0000","last_became_active":"2024-09-11T08:30:05.974284+0000","last_became_peered":"2024-09-11T08:30:05.974284+0000","last_unstale":"2024-09-11T08:30:14.679425+0000","last_undegraded":"2024-09-11T08:30:14.679425+0000","last_fullsized":"2024-09-11T08:30:14.679425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:39.981411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'2","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.680517+0000","last_change":"2024-09-11T08:30:02.998437+0000","last_active":"2024-09-11T08:30:14.680517+0000","last_peered":"2024-09-11T08:30:14.680517+0000","last_clean":"2024-09-11T08:30:14.680517+0000","last_became_active":"2024-09-11T08:30:02.998098+0000","last_became_peered":"2024-09-11T08:30:02.998098+0000","last_unstale":"2024-09-11T08:30:14.680517+0000","last_undegraded":"2024-09-11T08:30:14.680517+0000","last_fullsized":"2024-09-11T08:30:14.680517+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:28.205808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264331+0000","last_change":"2024-09-11T08:30:07.096855+0000","last_active":"2024-09-11T08:31:21.264331+0000","last_peered":"2024-09-11T08:31:21.264331+0000","last_clean":"2024-09-11T08:31:21.264331+0000","last_became_active":"2024-09-11T08:30:07.096521+0000","last_became_peered":"2024-09-11T08:30:07.096521+0000","last_unstale":"2024-09-11T08:31:21.264331+0000","last_undegraded":"2024-09-11T08:31:21.264331+0000","last_fullsized":"2024-09-11T08:31:21.264331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:56:50.703866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678609+0000","last_change":"2024-09-11T08:30:09.640736+0000","last_active":"2024-09-11T08:30:14.678609+0000","last_peered":"2024-09-11T08:30:14.678609+0000","last_clean":"2024-09-11T08:30:14.678609+0000","last_became_active":"2024-09-11T08:30:09.640413+0000","last_became_peered":"2024-09-11T08:30:09.640413+0000","last_unstale":"2024-09-11T08:30:14.678609+0000","last_undegraded":"2024-09-11T08:30:14.678609+0000","last_fullsized":"2024-09-11T08:30:14.678609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:11:26.409559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678590+0000","last_change":"2024-09-11T08:30:02.997909+0000","last_active":"2024-09-11T08:30:14.678590+0000","last_peered":"2024-09-11T08:30:14.678590+0000","last_clean":"2024-09-11T08:30:14.678590+0000","last_became_active":"2024-09-11T08:30:02.997606+0000","last_became_peered":"2024-09-11T08:30:02.997606+0000","last_unstale":"2024-09-11T08:30:14.678590+0000","last_undegraded":"2024-09-11T08:30:14.678590+0000","last_fullsized":"2024-09-11T08:30:14.678590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:25.397503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679319+0000","last_change":"2024-09-11T08:30:05.974206+0000","last_active":"2024-09-11T08:30:14.679319+0000","last_peered":"2024-09-11T08:30:14.679319+0000","last_clean":"2024-09-11T08:30:14.679319+0000","last_became_active":"2024-09-11T08:30:05.973567+0000","last_became_peered":"2024-09-11T08:30:05.973567+0000","last_unstale":"2024-09-11T08:30:14.679319+0000","last_undegraded":"2024-09-11T08:30:14.679319+0000","last_fullsized":"2024-09-11T08:30:14.679319+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:26.872664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679797+0000","last_change":"2024-09-11T08:30:07.091157+0000","last_active":"2024-09-11T08:30:14.679797+0000","last_peered":"2024-09-11T08:30:14.679797+0000","last_clean":"2024-09-11T08:30:14.679797+0000","last_became_active":"2024-09-11T08:30:07.090852+0000","last_became_peered":"2024-09-11T08:30:07.090852+0000","last_unstale":"2024-09-11T08:30:14.679797+0000","last_undegraded":"2024-09-11T08:30:14.679797+0000","last_fullsized":"2024-09-11T08:30:14.679797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:03.448113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678698+0000","last_change":"2024-09-11T08:30:09.641099+0000","last_active":"2024-09-11T08:30:14.678698+0000","last_peered":"2024-09-11T08:30:14.678698+0000","last_clean":"2024-09-11T08:30:14.678698+0000","last_became_active":"2024-09-11T08:30:09.640806+0000","last_became_peered":"2024-09-11T08:30:09.640806+0000","last_unstale":"2024-09-11T08:30:14.678698+0000","last_undegraded":"2024-09-11T08:30:14.678698+0000","last_fullsized":"2024-09-11T08:30:14.678698+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:04:22.496535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250267+0000","last_change":"2024-09-11T08:30:05.975575+0000","last_active":"2024-09-11T08:30:14.250267+0000","last_peered":"2024-09-11T08:30:14.250267+0000","last_clean":"2024-09-11T08:30:14.250267+0000","last_became_active":"2024-09-11T08:30:05.974386+0000","last_became_peered":"2024-09-11T08:30:05.974386+0000","last_unstale":"2024-09-11T08:30:14.250267+0000","last_undegraded":"2024-09-11T08:30:14.250267+0000","last_fullsized":"2024-09-11T08:30:14.250267+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:53.649662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678611+0000","last_change":"2024-09-11T08:30:02.999621+0000","last_active":"2024-09-11T08:30:14.678611+0000","last_peered":"2024-09-11T08:30:14.678611+0000","last_clean":"2024-09-11T08:30:14.678611+0000","last_became_active":"2024-09-11T08:30:02.998649+0000","last_became_peered":"2024-09-11T08:30:02.998649+0000","last_unstale":"2024-09-11T08:30:14.678611+0000","last_undegraded":"2024-09-11T08:30:14.678611+0000","last_fullsized":"2024-09-11T08:30:14.678611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:17:38.749919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679178+0000","last_change":"2024-09-11T08:30:07.093621+0000","last_active":"2024-09-11T08:30:14.679178+0000","last_peered":"2024-09-11T08:30:14.679178+0000","last_clean":"2024-09-11T08:30:14.679178+0000","last_became_active":"2024-09-11T08:30:07.092092+0000","last_became_peered":"2024-09-11T08:30:07.092092+0000","last_unstale":"2024-09-11T08:30:14.679178+0000","last_undegraded":"2024-09-11T08:30:14.679178+0000","last_fullsized":"2024-09-11T08:30:14.679178+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:31.169507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059733+0000","last_change":"2024-09-11T08:30:09.110871+0000","last_active":"2024-09-11T08:30:14.059733+0000","last_peered":"2024-09-11T08:30:14.059733+0000","last_clean":"2024-09-11T08:30:14.059733+0000","last_became_active":"2024-09-11T08:30:09.110548+0000","last_became_peered":"2024-09-11T08:30:09.110548+0000","last_unstale":"2024-09-11T08:30:14.059733+0000","last_undegraded":"2024-09-11T08:30:14.059733+0000","last_fullsized":"2024-09-11T08:30:14.059733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:08:01.967506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679795+0000","last_change":"2024-09-11T08:30:05.974817+0000","last_active":"2024-09-11T08:30:14.679795+0000","last_peered":"2024-09-11T08:30:14.679795+0000","last_clean":"2024-09-11T08:30:14.679795+0000","last_became_active":"2024-09-11T08:30:05.974399+0000","last_became_peered":"2024-09-11T08:30:05.974399+0000","last_unstale":"2024-09-11T08:30:14.679795+0000","last_undegraded":"2024-09-11T08:30:14.679795+0000","last_fullsized":"2024-09-11T08:30:14.679795+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:10.562163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249594+0000","last_change":"2024-09-11T08:30:02.992432+0000","last_active":"2024-09-11T08:30:14.249594+0000","last_peered":"2024-09-11T08:30:14.249594+0000","last_clean":"2024-09-11T08:30:14.249594+0000","last_became_active":"2024-09-11T08:30:02.992145+0000","last_became_peered":"2024-09-11T08:30:02.992145+0000","last_unstale":"2024-09-11T08:30:14.249594+0000","last_undegraded":"2024-09-11T08:30:14.249594+0000","last_fullsized":"2024-09-11T08:30:14.249594+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:42:14.459417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264274+0000","last_change":"2024-09-11T08:30:07.094423+0000","last_active":"2024-09-11T08:31:21.264274+0000","last_peered":"2024-09-11T08:31:21.264274+0000","last_clean":"2024-09-11T08:31:21.264274+0000","last_became_active":"2024-09-11T08:30:07.094081+0000","last_became_peered":"2024-09-11T08:30:07.094081+0000","last_unstale":"2024-09-11T08:31:21.264274+0000","last_undegraded":"2024-09-11T08:31:21.264274+0000","last_fullsized":"2024-09-11T08:31:21.264274+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:06:36.885569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059548+0000","last_change":"2024-09-11T08:30:09.111164+0000","last_active":"2024-09-11T08:30:14.059548+0000","last_peered":"2024-09-11T08:30:14.059548+0000","last_clean":"2024-09-11T08:30:14.059548+0000","last_became_active":"2024-09-11T08:30:09.110662+0000","last_became_peered":"2024-09-11T08:30:09.110662+0000","last_unstale":"2024-09-11T08:30:14.059548+0000","last_undegraded":"2024-09-11T08:30:14.059548+0000","last_fullsized":"2024-09-11T08:30:14.059548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:35.329576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679215+0000","last_change":"2024-09-11T08:30:05.973885+0000","last_active":"2024-09-11T08:30:14.679215+0000","last_peered":"2024-09-11T08:30:14.679215+0000","last_clean":"2024-09-11T08:30:14.679215+0000","last_became_active":"2024-09-11T08:30:05.973490+0000","last_became_peered":"2024-09-11T08:30:05.973490+0000","last_unstale":"2024-09-11T08:30:14.679215+0000","last_undegraded":"2024-09-11T08:30:14.679215+0000","last_fullsized":"2024-09-11T08:30:14.679215+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:08:05.884036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678773+0000","last_change":"2024-09-11T08:30:02.996121+0000","last_active":"2024-09-11T08:30:14.678773+0000","last_peered":"2024-09-11T08:30:14.678773+0000","last_clean":"2024-09-11T08:30:14.678773+0000","last_became_active":"2024-09-11T08:30:02.995828+0000","last_became_peered":"2024-09-11T08:30:02.995828+0000","last_unstale":"2024-09-11T08:30:14.678773+0000","last_undegraded":"2024-09-11T08:30:14.678773+0000","last_fullsized":"2024-09-11T08:30:14.678773+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:05.529343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264329+0000","last_change":"2024-09-11T08:30:07.100063+0000","last_active":"2024-09-11T08:31:21.264329+0000","last_peered":"2024-09-11T08:31:21.264329+0000","last_clean":"2024-09-11T08:31:21.264329+0000","last_became_active":"2024-09-11T08:30:07.099752+0000","last_became_peered":"2024-09-11T08:30:07.099752+0000","last_unstale":"2024-09-11T08:31:21.264329+0000","last_undegraded":"2024-09-11T08:31:21.264329+0000","last_fullsized":"2024-09-11T08:31:21.264329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:04:19.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250017+0000","last_change":"2024-09-11T08:30:09.114755+0000","last_active":"2024-09-11T08:30:14.250017+0000","last_peered":"2024-09-11T08:30:14.250017+0000","last_clean":"2024-09-11T08:30:14.250017+0000","last_became_active":"2024-09-11T08:30:09.114428+0000","last_became_peered":"2024-09-11T08:30:09.114428+0000","last_unstale":"2024-09-11T08:30:14.250017+0000","last_undegraded":"2024-09-11T08:30:14.250017+0000","last_fullsized":"2024-09-11T08:30:14.250017+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:21.459812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059036+0000","last_change":"2024-09-11T08:30:05.080269+0000","last_active":"2024-09-11T08:30:14.059036+0000","last_peered":"2024-09-11T08:30:14.059036+0000","last_clean":"2024-09-11T08:30:14.059036+0000","last_became_active":"2024-09-11T08:30:05.079754+0000","last_became_peered":"2024-09-11T08:30:05.079754+0000","last_unstale":"2024-09-11T08:30:14.059036+0000","last_undegraded":"2024-09-11T08:30:14.059036+0000","last_fullsized":"2024-09-11T08:30:14.059036+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:27.996903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"53'1","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059266+0000","last_change":"2024-09-11T08:30:02.996409+0000","last_active":"2024-09-11T08:30:14.059266+0000","last_peered":"2024-09-11T08:30:14.059266+0000","last_clean":"2024-09-11T08:30:14.059266+0000","last_became_active":"2024-09-11T08:30:02.996127+0000","last_became_peered":"2024-09-11T08:30:02.996127+0000","last_unstale":"2024-09-11T08:30:14.059266+0000","last_undegraded":"2024-09-11T08:30:14.059266+0000","last_fullsized":"2024-09-11T08:30:14.059266+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:12.139498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059742+0000","last_change":"2024-09-11T08:30:07.090330+0000","last_active":"2024-09-11T08:30:14.059742+0000","last_peered":"2024-09-11T08:30:14.059742+0000","last_clean":"2024-09-11T08:30:14.059742+0000","last_became_active":"2024-09-11T08:30:07.088941+0000","last_became_peered":"2024-09-11T08:30:07.088941+0000","last_unstale":"2024-09-11T08:30:14.059742+0000","last_undegraded":"2024-09-11T08:30:14.059742+0000","last_fullsized":"2024-09-11T08:30:14.059742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:24.899987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678194+0000","last_change":"2024-09-11T08:30:09.113876+0000","last_active":"2024-09-11T08:30:14.678194+0000","last_peered":"2024-09-11T08:30:14.678194+0000","last_clean":"2024-09-11T08:30:14.678194+0000","last_became_active":"2024-09-11T08:30:09.113500+0000","last_became_peered":"2024-09-11T08:30:09.113500+0000","last_unstale":"2024-09-11T08:30:14.678194+0000","last_undegraded":"2024-09-11T08:30:14.678194+0000","last_fullsized":"2024-09-11T08:30:14.678194+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:47.202797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060024+0000","last_change":"2024-09-11T08:30:05.070804+0000","last_active":"2024-09-11T08:30:14.060024+0000","last_peered":"2024-09-11T08:30:14.060024+0000","last_clean":"2024-09-11T08:30:14.060024+0000","last_became_active":"2024-09-11T08:30:05.070470+0000","last_became_peered":"2024-09-11T08:30:05.070470+0000","last_unstale":"2024-09-11T08:30:14.060024+0000","last_undegraded":"2024-09-11T08:30:14.060024+0000","last_fullsized":"2024-09-11T08:30:14.060024+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:35.006562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678979+0000","last_change":"2024-09-11T08:30:02.996378+0000","last_active":"2024-09-11T08:30:14.678979+0000","last_peered":"2024-09-11T08:30:14.678979+0000","last_clean":"2024-09-11T08:30:14.678979+0000","last_became_active":"2024-09-11T08:30:02.996118+0000","last_became_peered":"2024-09-11T08:30:02.996118+0000","last_unstale":"2024-09-11T08:30:14.678979+0000","last_undegraded":"2024-09-11T08:30:14.678979+0000","last_fullsized":"2024-09-11T08:30:14.678979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:34:39.261710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678418+0000","last_change":"2024-09-11T08:30:07.098119+0000","last_active":"2024-09-11T08:30:14.678418+0000","last_peered":"2024-09-11T08:30:14.678418+0000","last_clean":"2024-09-11T08:30:14.678418+0000","last_became_active":"2024-09-11T08:30:07.097776+0000","last_became_peered":"2024-09-11T08:30:07.097776+0000","last_unstale":"2024-09-11T08:30:14.678418+0000","last_undegraded":"2024-09-11T08:30:14.678418+0000","last_fullsized":"2024-09-11T08:30:14.678418+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:54.678730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249324+0000","last_change":"2024-09-11T08:30:09.112757+0000","last_active":"2024-09-11T08:30:14.249324+0000","last_peered":"2024-09-11T08:30:14.249324+0000","last_clean":"2024-09-11T08:30:14.249324+0000","last_became_active":"2024-09-11T08:30:09.112410+0000","last_became_peered":"2024-09-11T08:30:09.112410+0000","last_unstale":"2024-09-11T08:30:14.249324+0000","last_undegraded":"2024-09-11T08:30:14.249324+0000","last_fullsized":"2024-09-11T08:30:14.249324+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:56.666041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678358+0000","last_change":"2024-09-11T08:30:05.974060+0000","last_active":"2024-09-11T08:30:14.678358+0000","last_peered":"2024-09-11T08:30:14.678358+0000","last_clean":"2024-09-11T08:30:14.678358+0000","last_became_active":"2024-09-11T08:30:05.973805+0000","last_became_peered":"2024-09-11T08:30:05.973805+0000","last_unstale":"2024-09-11T08:30:14.678358+0000","last_undegraded":"2024-09-11T08:30:14.678358+0000","last_fullsized":"2024-09-11T08:30:14.678358+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:20:54.951937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679846+0000","last_change":"2024-09-11T08:30:02.991063+0000","last_active":"2024-09-11T08:30:14.679846+0000","last_peered":"2024-09-11T08:30:14.679846+0000","last_clean":"2024-09-11T08:30:14.679846+0000","last_became_active":"2024-09-11T08:30:02.990625+0000","last_became_peered":"2024-09-11T08:30:02.990625+0000","last_unstale":"2024-09-11T08:30:14.679846+0000","last_undegraded":"2024-09-11T08:30:14.679846+0000","last_fullsized":"2024-09-11T08:30:14.679846+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:07:12.143627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678036+0000","last_change":"2024-09-11T08:30:07.094339+0000","last_active":"2024-09-11T08:30:14.678036+0000","last_peered":"2024-09-11T08:30:14.678036+0000","last_clean":"2024-09-11T08:30:14.678036+0000","last_became_active":"2024-09-11T08:30:07.094034+0000","last_became_peered":"2024-09-11T08:30:07.094034+0000","last_unstale":"2024-09-11T08:30:14.678036+0000","last_undegraded":"2024-09-11T08:30:14.678036+0000","last_fullsized":"2024-09-11T08:30:14.678036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:47.199571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678979+0000","last_change":"2024-09-11T08:30:09.643435+0000","last_active":"2024-09-11T08:30:14.678979+0000","last_peered":"2024-09-11T08:30:14.678979+0000","last_clean":"2024-09-11T08:30:14.678979+0000","last_became_active":"2024-09-11T08:30:09.643128+0000","last_became_peered":"2024-09-11T08:30:09.643128+0000","last_unstale":"2024-09-11T08:30:14.678979+0000","last_undegraded":"2024-09-11T08:30:14.678979+0000","last_fullsized":"2024-09-11T08:30:14.678979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:04.979550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678192+0000","last_change":"2024-09-11T08:30:05.080210+0000","last_active":"2024-09-11T08:30:14.678192+0000","last_peered":"2024-09-11T08:30:14.678192+0000","last_clean":"2024-09-11T08:30:14.678192+0000","last_became_active":"2024-09-11T08:30:05.079930+0000","last_became_peered":"2024-09-11T08:30:05.079930+0000","last_unstale":"2024-09-11T08:30:14.678192+0000","last_undegraded":"2024-09-11T08:30:14.678192+0000","last_fullsized":"2024-09-11T08:30:14.678192+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:54.261529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250346+0000","last_change":"2024-09-11T08:30:02.995939+0000","last_active":"2024-09-11T08:30:14.250346+0000","last_peered":"2024-09-11T08:30:14.250346+0000","last_clean":"2024-09-11T08:30:14.250346+0000","last_became_active":"2024-09-11T08:30:02.995620+0000","last_became_peered":"2024-09-11T08:30:02.995620+0000","last_unstale":"2024-09-11T08:30:14.250346+0000","last_undegraded":"2024-09-11T08:30:14.250346+0000","last_fullsized":"2024-09-11T08:30:14.250346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:46.695273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678276+0000","last_change":"2024-09-11T08:30:07.099011+0000","last_active":"2024-09-11T08:30:14.678276+0000","last_peered":"2024-09-11T08:30:14.678276+0000","last_clean":"2024-09-11T08:30:14.678276+0000","last_became_active":"2024-09-11T08:30:07.098633+0000","last_became_peered":"2024-09-11T08:30:07.098633+0000","last_unstale":"2024-09-11T08:30:14.678276+0000","last_undegraded":"2024-09-11T08:30:14.678276+0000","last_fullsized":"2024-09-11T08:30:14.678276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:06.219514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678426+0000","last_change":"2024-09-11T08:30:09.109045+0000","last_active":"2024-09-11T08:30:14.678426+0000","last_peered":"2024-09-11T08:30:14.678426+0000","last_clean":"2024-09-11T08:30:14.678426+0000","last_became_active":"2024-09-11T08:30:09.108734+0000","last_became_peered":"2024-09-11T08:30:09.108734+0000","last_unstale":"2024-09-11T08:30:14.678426+0000","last_undegraded":"2024-09-11T08:30:14.678426+0000","last_fullsized":"2024-09-11T08:30:14.678426+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:41.055857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060031+0000","last_change":"2024-09-11T08:30:09.102625+0000","last_active":"2024-09-11T08:30:14.060031+0000","last_peered":"2024-09-11T08:30:14.060031+0000","last_clean":"2024-09-11T08:30:14.060031+0000","last_became_active":"2024-09-11T08:30:09.102309+0000","last_became_peered":"2024-09-11T08:30:09.102309+0000","last_unstale":"2024-09-11T08:30:14.060031+0000","last_undegraded":"2024-09-11T08:30:14.060031+0000","last_fullsized":"2024-09-11T08:30:14.060031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:04.315237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"53'1","reported_seq":27,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678388+0000","last_change":"2024-09-11T08:30:02.997681+0000","last_active":"2024-09-11T08:30:14.678388+0000","last_peered":"2024-09-11T08:30:14.678388+0000","last_clean":"2024-09-11T08:30:14.678388+0000","last_became_active":"2024-09-11T08:30:02.997422+0000","last_became_peered":"2024-09-11T08:30:02.997422+0000","last_unstale":"2024-09-11T08:30:14.678388+0000","last_undegraded":"2024-09-11T08:30:14.678388+0000","last_fullsized":"2024-09-11T08:30:14.678388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:26.866200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679489+0000","last_change":"2024-09-11T08:30:05.082897+0000","last_active":"2024-09-11T08:30:14.679489+0000","last_peered":"2024-09-11T08:30:14.679489+0000","last_clean":"2024-09-11T08:30:14.679489+0000","last_became_active":"2024-09-11T08:30:05.082529+0000","last_became_peered":"2024-09-11T08:30:05.082529+0000","last_unstale":"2024-09-11T08:30:14.679489+0000","last_undegraded":"2024-09-11T08:30:14.679489+0000","last_fullsized":"2024-09-11T08:30:14.679489+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:36.776919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250045+0000","last_change":"2024-09-11T08:30:07.097926+0000","last_active":"2024-09-11T08:30:14.250045+0000","last_peered":"2024-09-11T08:30:14.250045+0000","last_clean":"2024-09-11T08:30:14.250045+0000","last_became_active":"2024-09-11T08:30:07.097600+0000","last_became_peered":"2024-09-11T08:30:07.097600+0000","last_unstale":"2024-09-11T08:30:14.250045+0000","last_undegraded":"2024-09-11T08:30:14.250045+0000","last_fullsized":"2024-09-11T08:30:14.250045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:07:52.470385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":73,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397526,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298548,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446604,"statfs":{"total":95995035648,"available":95689322496,"internally_reserved":0,"allocated":1224704,"data_stored":825707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.56699999999999995}]},{"osd":1,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.499}]},{"osd":2,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.45000000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.621}]},{"osd":6,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.48799999999999999}]}]},{"osd":6,"up_from":43,"seq":184683593755,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298520,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446632,"statfs":{"total":95995035648,"available":95689351168,"internally_reserved":0,"allocated":1196032,"data_stored":824097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.44}]},{"osd":1,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.434,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74299999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.47099999999999997}]},{"osd":4,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.376},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.42599999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.41999999999999998,"15min":0.40200000000000002},"min":{"1min":0.32800000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.35499999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.45300000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789985,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297944,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447208,"statfs":{"total":95995035648,"available":95689940992,"internally_reserved":0,"allocated":602112,"data_stored":233923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.70999999999999996,"5min":0.87,"15min":0.87},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81699999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82699999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55000000000000004}]},{"osd":3,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41299999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.32700000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40899999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297960,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447192,"statfs":{"total":95995035648,"available":95689924608,"internally_reserved":0,"allocated":618496,"data_stored":234208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66500000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.71299999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.51800000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:31:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.29199999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.378},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.47799999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.432}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":235645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36899999999999999,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52800000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.308},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.28799999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.41199999999999998,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.48899999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.39200000000000002,"15min":0.40200000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.38300000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.50600000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34200000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80600000000000005,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51700000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67200000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.50900000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.52700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69099999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.69299999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:30:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.434},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48099999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":234027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35999999999999999,"15min":0.35399999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.48799999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.375},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.375,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.31}]},{"osd":1,"last update":"Wed Sep 11 08:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.26600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41799999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65100000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.28499999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46000000000000002,"15min":0.47299999999999998},"min":{"1min":0.29699999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.64200000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.48099999999999998,"15min":0.498},"min":{"1min":0.32200000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65900000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.33300000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65100000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41199999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.434},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.70399999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45800000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:30:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":663552,"data_stored":236480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.377,"15min":0.34499999999999997},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.433,"15min":0.39000000000000001},"min":{"1min":0.249,"5min":0.218,"15min":0.218},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.39200000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.443},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59199999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.31,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64400000000000002,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.31}]},{"osd":4,"last update":"Wed Sep 11 08:30:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.501},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79900000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86499999999999999,"5min":1.482,"15min":1.482},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74299999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Sep 11 08:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.63100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1220608,"data_stored":825903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.32100000000000001,"15min":0.29399999999999998},"min":{"1min":0.248,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54300000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.32700000000000001,"15min":0.30399999999999999},"min":{"1min":0.26200000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.24199999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.25600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.41299999999999998,"15min":0.39900000000000002},"min":{"1min":0.32600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:31:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.53200000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32200000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:30:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.441},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.39400000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.41899999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42599999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60099999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.36799999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:31:29.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:31:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:30.574 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:30 smithi043 ceph-mon[40592]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:31:30.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:30 smithi043 ceph-mon[29355]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:31:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:30 smithi172 ceph-mon[32987]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:31:31.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:31 smithi043 ceph-mon[40592]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:31.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:31 smithi043 ceph-mon[29355]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:31.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.956+0000 7fd920964640 1 -- 172.21.15.43:0/1556300004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fc0e0 msgr2=0x7fd91c108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:31.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.956+0000 7fd920964640 1 --2- 172.21.15.43:0/1556300004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fc0e0 0x7fd91c108500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9140048c0 tx=0x7fd914009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.956+0000 7fd920964640 1 -- 172.21.15.43:0/1556300004 shutdown_connections 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.956+0000 7fd920964640 1 --2- 172.21.15.43:0/1556300004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd91c0fb730 0x7fd91c0fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.956+0000 7fd920964640 1 --2- 172.21.15.43:0/1556300004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fc0e0 0x7fd91c108500 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.956+0000 7fd920964640 1 --2- 172.21.15.43:0/1556300004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd91c101750 0x7fd91c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.956+0000 7fd920964640 1 -- 172.21.15.43:0/1556300004 >> 172.21.15.43:0/1556300004 conn(0x7fd91c072580 msgr2=0x7fd91c0f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 -- 172.21.15.43:0/1556300004 shutdown_connections 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 -- 172.21.15.43:0/1556300004 wait complete. 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 Processor -- start 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 -- start start 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fb730 0x7fd91c112a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd91c0fc0e0 0x7fd91c112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd91c101750 0x7fd91c10cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c10d220 con 0x7fd91c0fb730 2024-09-11T08:31:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c10d390 con 0x7fd91c0fc0e0 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd920964640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c10d550 con 0x7fd91c101750 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fb730 0x7fd91c112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd91ad9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd91c101750 0x7fd91c10cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fb730 0x7fd91c112a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53540/0 (socket says 172.21.15.43:53540) 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.957+0000 7fd91a59c640 1 -- 172.21.15.43:0/3099550897 learned_addr learned my addr 172.21.15.43:0/3099550897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd919d9b640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd91c0fc0e0 0x7fd91c112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd91a59c640 1 -- 172.21.15.43:0/3099550897 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd91c101750 msgr2=0x7fd91c10cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd91a59c640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd91c101750 0x7fd91c10cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd91a59c640 1 -- 172.21.15.43:0/3099550897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd91c0fc0e0 msgr2=0x7fd91c112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd91a59c640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd91c0fc0e0 0x7fd91c112f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd91a59c640 1 -- 172.21.15.43:0/3099550897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910005040 con 0x7fd91c0fb730 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd91ad9d640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd91c101750 0x7fd91c10cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd91a59c640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fb730 0x7fd91c112a40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00dc00 tx=0x7fd90c004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:31.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd90affd640 1 -- 172.21.15.43:0/3099550897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c017070 con 0x7fd91c0fb730 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd90affd640 1 -- 172.21.15.43:0/3099550897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd90c00f040 con 0x7fd91c0fb730 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd914005040 con 0x7fd91c0fb730 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd90affd640 1 -- 172.21.15.43:0/3099550897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0037c0 con 0x7fd91c0fb730 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.958+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c119890 con 0x7fd91c0fb730 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.959+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c0fce70 con 0x7fd91c0fb730 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.960+0000 7fd90affd640 1 -- 172.21.15.43:0/3099550897 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fd90c00b9d0 con 0x7fd91c0fb730 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.960+0000 7fd90affd640 1 --2- 172.21.15.43:0/3099550897 >> v2:172.21.15.43:6800/588375333 conn(0x7fd90406ef80 0x7fd904071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.960+0000 7fd919d9b640 1 --2- 172.21.15.43:0/3099550897 >> v2:172.21.15.43:6800/588375333 conn(0x7fd90406ef80 0x7fd904071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.960+0000 7fd90affd640 1 -- 172.21.15.43:0/3099550897 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7fd90c08e4a0 con 0x7fd91c0fb730 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.961+0000 7fd919d9b640 1 --2- 172.21.15.43:0/3099550897 >> v2:172.21.15.43:6800/588375333 conn(0x7fd90406ef80 0x7fd904071430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd910007fd0 tx=0x7fd91000b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:31.964+0000 7fd90affd640 1 -- 172.21.15.43:0/3099550897 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd90c05b310 con 0x7fd91c0fb730 2024-09-11T08:31:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:31 smithi172 ceph-mon[32987]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:32.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.126+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 --> v2:172.21.15.43:6800/588375333 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd91c10abe0 con 0x7fd90406ef80 2024-09-11T08:31:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.137+0000 7fd90affd640 1 -- 172.21.15.43:0/3099550897 <== mgr.14553 v2:172.21.15.43:6800/588375333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384659 (secure 0 0 0) 0x7fd91c10abe0 con 0x7fd90406ef80 2024-09-11T08:31:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:31:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 >> v2:172.21.15.43:6800/588375333 conn(0x7fd90406ef80 msgr2=0x7fd904071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 --2- 172.21.15.43:0/3099550897 >> v2:172.21.15.43:6800/588375333 conn(0x7fd90406ef80 0x7fd904071430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd910007fd0 tx=0x7fd91000b040 comp rx=0 tx=0).stop 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fb730 msgr2=0x7fd91c112a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fb730 0x7fd91c112a40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00dc00 tx=0x7fd90c004f00 comp rx=0 tx=0).stop 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 shutdown_connections 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 --2- 172.21.15.43:0/3099550897 >> v2:172.21.15.43:6800/588375333 conn(0x7fd90406ef80 0x7fd904071430 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd91c101750 0x7fd91c10cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd91c0fc0e0 0x7fd91c112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 --2- 172.21.15.43:0/3099550897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91c0fb730 0x7fd91c112a40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 >> 172.21.15.43:0/3099550897 conn(0x7fd91c072580 msgr2=0x7fd91c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 shutdown_connections 2024-09-11T08:31:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:32.142+0000 7fd920964640 1 -- 172.21.15.43:0/3099550897 wait complete. 2024-09-11T08:31:32.886 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-11T08:31:30.164692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":931,"num_read_kb":788,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385444,"kb_used_data":6608,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575772,"statfs":{"total":767960285184,"available":765517590528,"internally_reserved":0,"allocated":6766592,"data_stored":3649990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002882"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250295+0000","last_change":"2024-09-11T08:30:02.996268+0000","last_active":"2024-09-11T08:30:14.250295+0000","last_peered":"2024-09-11T08:30:14.250295+0000","last_clean":"2024-09-11T08:30:14.250295+0000","last_became_active":"2024-09-11T08:30:02.995697+0000","last_became_peered":"2024-09-11T08:30:02.995697+0000","last_unstale":"2024-09-11T08:30:14.250295+0000","last_undegraded":"2024-09-11T08:30:14.250295+0000","last_fullsized":"2024-09-11T08:30:14.250295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:55.831568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058698+0000","last_change":"2024-09-11T08:30:05.070463+0000","last_active":"2024-09-11T08:30:14.058698+0000","last_peered":"2024-09-11T08:30:14.058698+0000","last_clean":"2024-09-11T08:30:14.058698+0000","last_became_active":"2024-09-11T08:30:05.070160+0000","last_became_peered":"2024-09-11T08:30:05.070160+0000","last_unstale":"2024-09-11T08:30:14.058698+0000","last_undegraded":"2024-09-11T08:30:14.058698+0000","last_fullsized":"2024-09-11T08:30:14.058698+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:41.965422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.061030+0000","last_change":"2024-09-11T08:30:07.090704+0000","last_active":"2024-09-11T08:30:14.061030+0000","last_peered":"2024-09-11T08:30:14.061030+0000","last_clean":"2024-09-11T08:30:14.061030+0000","last_became_active":"2024-09-11T08:30:07.089958+0000","last_became_peered":"2024-09-11T08:30:07.089958+0000","last_unstale":"2024-09-11T08:30:14.061030+0000","last_undegraded":"2024-09-11T08:30:14.061030+0000","last_fullsized":"2024-09-11T08:30:14.061030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:31.655881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058850+0000","last_change":"2024-09-11T08:30:09.102309+0000","last_active":"2024-09-11T08:30:14.058850+0000","last_peered":"2024-09-11T08:30:14.058850+0000","last_clean":"2024-09-11T08:30:14.058850+0000","last_became_active":"2024-09-11T08:30:09.102007+0000","last_became_peered":"2024-09-11T08:30:09.102007+0000","last_unstale":"2024-09-11T08:30:14.058850+0000","last_undegraded":"2024-09-11T08:30:14.058850+0000","last_fullsized":"2024-09-11T08:30:14.058850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:07:50.975901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678270+0000","last_change":"2024-09-11T08:30:09.643443+0000","last_active":"2024-09-11T08:30:14.678270+0000","last_peered":"2024-09-11T08:30:14.678270+0000","last_clean":"2024-09-11T08:30:14.678270+0000","last_became_active":"2024-09-11T08:30:09.643122+0000","last_became_peered":"2024-09-11T08:30:09.643122+0000","last_unstale":"2024-09-11T08:30:14.678270+0000","last_undegraded":"2024-09-11T08:30:14.678270+0000","last_fullsized":"2024-09-11T08:30:14.678270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:55.728852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679045+0000","last_change":"2024-09-11T08:30:02.986951+0000","last_active":"2024-09-11T08:30:14.679045+0000","last_peered":"2024-09-11T08:30:14.679045+0000","last_clean":"2024-09-11T08:30:14.679045+0000","last_became_active":"2024-09-11T08:30:02.986647+0000","last_became_peered":"2024-09-11T08:30:02.986647+0000","last_unstale":"2024-09-11T08:30:14.679045+0000","last_undegraded":"2024-09-11T08:30:14.679045+0000","last_fullsized":"2024-09-11T08:30:14.679045+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:52.161933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678533+0000","last_change":"2024-09-11T08:30:05.077989+0000","last_active":"2024-09-11T08:30:14.678533+0000","last_peered":"2024-09-11T08:30:14.678533+0000","last_clean":"2024-09-11T08:30:14.678533+0000","last_became_active":"2024-09-11T08:30:05.076716+0000","last_became_peered":"2024-09-11T08:30:05.076716+0000","last_unstale":"2024-09-11T08:30:14.678533+0000","last_undegraded":"2024-09-11T08:30:14.678533+0000","last_fullsized":"2024-09-11T08:30:14.678533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:32.636665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058694+0000","last_change":"2024-09-11T08:30:07.094953+0000","last_active":"2024-09-11T08:30:14.058694+0000","last_peered":"2024-09-11T08:30:14.058694+0000","last_clean":"2024-09-11T08:30:14.058694+0000","last_became_active":"2024-09-11T08:30:07.094482+0000","last_became_peered":"2024-09-11T08:30:07.094482+0000","last_unstale":"2024-09-11T08:30:14.058694+0000","last_undegraded":"2024-09-11T08:30:14.058694+0000","last_fullsized":"2024-09-11T08:30:14.058694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:43:53.526055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679846+0000","last_change":"2024-09-11T08:30:02.991821+0000","last_active":"2024-09-11T08:30:14.679846+0000","last_peered":"2024-09-11T08:30:14.679846+0000","last_clean":"2024-09-11T08:30:14.679846+0000","last_became_active":"2024-09-11T08:30:02.991542+0000","last_became_peered":"2024-09-11T08:30:02.991542+0000","last_unstale":"2024-09-11T08:30:14.679846+0000","last_undegraded":"2024-09-11T08:30:14.679846+0000","last_fullsized":"2024-09-11T08:30:14.679846+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:11.128311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059007+0000","last_change":"2024-09-11T08:30:05.075388+0000","last_active":"2024-09-11T08:30:14.059007+0000","last_peered":"2024-09-11T08:30:14.059007+0000","last_clean":"2024-09-11T08:30:14.059007+0000","last_became_active":"2024-09-11T08:30:05.075107+0000","last_became_peered":"2024-09-11T08:30:05.075107+0000","last_unstale":"2024-09-11T08:30:14.059007+0000","last_undegraded":"2024-09-11T08:30:14.059007+0000","last_fullsized":"2024-09-11T08:30:14.059007+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:43:37.495052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679231+0000","last_change":"2024-09-11T08:30:07.100815+0000","last_active":"2024-09-11T08:30:14.679231+0000","last_peered":"2024-09-11T08:30:14.679231+0000","last_clean":"2024-09-11T08:30:14.679231+0000","last_became_active":"2024-09-11T08:30:07.100134+0000","last_became_peered":"2024-09-11T08:30:07.100134+0000","last_unstale":"2024-09-11T08:30:14.679231+0000","last_undegraded":"2024-09-11T08:30:14.679231+0000","last_fullsized":"2024-09-11T08:30:14.679231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:27.216340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249440+0000","last_change":"2024-09-11T08:30:09.114943+0000","last_active":"2024-09-11T08:30:14.249440+0000","last_peered":"2024-09-11T08:30:14.249440+0000","last_clean":"2024-09-11T08:30:14.249440+0000","last_became_active":"2024-09-11T08:30:09.114511+0000","last_became_peered":"2024-09-11T08:30:09.114511+0000","last_unstale":"2024-09-11T08:30:14.249440+0000","last_undegraded":"2024-09-11T08:30:14.249440+0000","last_fullsized":"2024-09-11T08:30:14.249440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:30:32.217603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679565+0000","last_change":"2024-09-11T08:30:02.994076+0000","last_active":"2024-09-11T08:30:14.679565+0000","last_peered":"2024-09-11T08:30:14.679565+0000","last_clean":"2024-09-11T08:30:14.679565+0000","last_became_active":"2024-09-11T08:30:02.993609+0000","last_became_peered":"2024-09-11T08:30:02.993609+0000","last_unstale":"2024-09-11T08:30:14.679565+0000","last_undegraded":"2024-09-11T08:30:14.679565+0000","last_fullsized":"2024-09-11T08:30:14.679565+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:29.826139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":26,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058989+0000","last_change":"2024-09-11T08:30:05.073968+0000","last_active":"2024-09-11T08:30:14.058989+0000","last_peered":"2024-09-11T08:30:14.058989+0000","last_clean":"2024-09-11T08:30:14.058989+0000","last_became_active":"2024-09-11T08:30:05.073635+0000","last_became_peered":"2024-09-11T08:30:05.073635+0000","last_unstale":"2024-09-11T08:30:14.058989+0000","last_undegraded":"2024-09-11T08:30:14.058989+0000","last_fullsized":"2024-09-11T08:30:14.058989+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:58:48.408406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678500+0000","last_change":"2024-09-11T08:30:07.092049+0000","last_active":"2024-09-11T08:30:14.678500+0000","last_peered":"2024-09-11T08:30:14.678500+0000","last_clean":"2024-09-11T08:30:14.678500+0000","last_became_active":"2024-09-11T08:30:07.091750+0000","last_became_peered":"2024-09-11T08:30:07.091750+0000","last_unstale":"2024-09-11T08:30:14.678500+0000","last_undegraded":"2024-09-11T08:30:14.678500+0000","last_fullsized":"2024-09-11T08:30:14.678500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:20.899947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678422+0000","last_change":"2024-09-11T08:30:09.105281+0000","last_active":"2024-09-11T08:30:14.678422+0000","last_peered":"2024-09-11T08:30:14.678422+0000","last_clean":"2024-09-11T08:30:14.678422+0000","last_became_active":"2024-09-11T08:30:09.104983+0000","last_became_peered":"2024-09-11T08:30:09.104983+0000","last_unstale":"2024-09-11T08:30:14.678422+0000","last_undegraded":"2024-09-11T08:30:14.678422+0000","last_fullsized":"2024-09-11T08:30:14.678422+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:38:35.126318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059323+0000","last_change":"2024-09-11T08:30:09.641951+0000","last_active":"2024-09-11T08:30:14.059323+0000","last_peered":"2024-09-11T08:30:14.059323+0000","last_clean":"2024-09-11T08:30:14.059323+0000","last_became_active":"2024-09-11T08:30:09.640951+0000","last_became_peered":"2024-09-11T08:30:09.640951+0000","last_unstale":"2024-09-11T08:30:14.059323+0000","last_undegraded":"2024-09-11T08:30:14.059323+0000","last_fullsized":"2024-09-11T08:30:14.059323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:51.992594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250303+0000","last_change":"2024-09-11T08:30:02.996395+0000","last_active":"2024-09-11T08:30:14.250303+0000","last_peered":"2024-09-11T08:30:14.250303+0000","last_clean":"2024-09-11T08:30:14.250303+0000","last_became_active":"2024-09-11T08:30:02.996121+0000","last_became_peered":"2024-09-11T08:30:02.996121+0000","last_unstale":"2024-09-11T08:30:14.250303+0000","last_undegraded":"2024-09-11T08:30:14.250303+0000","last_fullsized":"2024-09-11T08:30:14.250303+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:04:57.548246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060170+0000","last_change":"2024-09-11T08:30:05.075898+0000","last_active":"2024-09-11T08:30:14.060170+0000","last_peered":"2024-09-11T08:30:14.060170+0000","last_clean":"2024-09-11T08:30:14.060170+0000","last_became_active":"2024-09-11T08:30:05.075562+0000","last_became_peered":"2024-09-11T08:30:05.075562+0000","last_unstale":"2024-09-11T08:30:14.060170+0000","last_undegraded":"2024-09-11T08:30:14.060170+0000","last_fullsized":"2024-09-11T08:30:14.060170+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:25.617204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060324+0000","last_change":"2024-09-11T08:30:07.101087+0000","last_active":"2024-09-11T08:30:14.060324+0000","last_peered":"2024-09-11T08:30:14.060324+0000","last_clean":"2024-09-11T08:30:14.060324+0000","last_became_active":"2024-09-11T08:30:07.099930+0000","last_became_peered":"2024-09-11T08:30:07.099930+0000","last_unstale":"2024-09-11T08:30:14.060324+0000","last_undegraded":"2024-09-11T08:30:14.060324+0000","last_fullsized":"2024-09-11T08:30:14.060324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:51:07.366130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678960+0000","last_change":"2024-09-11T08:30:09.641223+0000","last_active":"2024-09-11T08:30:14.678960+0000","last_peered":"2024-09-11T08:30:14.678960+0000","last_clean":"2024-09-11T08:30:14.678960+0000","last_became_active":"2024-09-11T08:30:09.640687+0000","last_became_peered":"2024-09-11T08:30:09.640687+0000","last_unstale":"2024-09-11T08:30:14.678960+0000","last_undegraded":"2024-09-11T08:30:14.678960+0000","last_fullsized":"2024-09-11T08:30:14.678960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:20:57.664659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059079+0000","last_change":"2024-09-11T08:30:02.994123+0000","last_active":"2024-09-11T08:30:14.059079+0000","last_peered":"2024-09-11T08:30:14.059079+0000","last_clean":"2024-09-11T08:30:14.059079+0000","last_became_active":"2024-09-11T08:30:02.993832+0000","last_became_peered":"2024-09-11T08:30:02.993832+0000","last_unstale":"2024-09-11T08:30:14.059079+0000","last_undegraded":"2024-09-11T08:30:14.059079+0000","last_fullsized":"2024-09-11T08:30:14.059079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:09.219971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678979+0000","last_change":"2024-09-11T08:30:05.084156+0000","last_active":"2024-09-11T08:30:14.678979+0000","last_peered":"2024-09-11T08:30:14.678979+0000","last_clean":"2024-09-11T08:30:14.678979+0000","last_became_active":"2024-09-11T08:30:05.083863+0000","last_became_peered":"2024-09-11T08:30:05.083863+0000","last_unstale":"2024-09-11T08:30:14.678979+0000","last_undegraded":"2024-09-11T08:30:14.678979+0000","last_fullsized":"2024-09-11T08:30:14.678979+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:56.915638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059079+0000","last_change":"2024-09-11T08:30:07.093294+0000","last_active":"2024-09-11T08:30:14.059079+0000","last_peered":"2024-09-11T08:30:14.059079+0000","last_clean":"2024-09-11T08:30:14.059079+0000","last_became_active":"2024-09-11T08:30:07.092984+0000","last_became_peered":"2024-09-11T08:30:07.092984+0000","last_unstale":"2024-09-11T08:30:14.059079+0000","last_undegraded":"2024-09-11T08:30:14.059079+0000","last_fullsized":"2024-09-11T08:30:14.059079+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:25:42.406618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678920+0000","last_change":"2024-09-11T08:30:09.111012+0000","last_active":"2024-09-11T08:30:14.678920+0000","last_peered":"2024-09-11T08:30:14.678920+0000","last_clean":"2024-09-11T08:30:14.678920+0000","last_became_active":"2024-09-11T08:30:09.110669+0000","last_became_peered":"2024-09-11T08:30:09.110669+0000","last_unstale":"2024-09-11T08:30:14.678920+0000","last_undegraded":"2024-09-11T08:30:14.678920+0000","last_fullsized":"2024-09-11T08:30:14.678920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:54:05.014760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.061119+0000","last_change":"2024-09-11T08:30:02.992025+0000","last_active":"2024-09-11T08:30:14.061119+0000","last_peered":"2024-09-11T08:30:14.061119+0000","last_clean":"2024-09-11T08:30:14.061119+0000","last_became_active":"2024-09-11T08:30:02.991364+0000","last_became_peered":"2024-09-11T08:30:02.991364+0000","last_unstale":"2024-09-11T08:30:14.061119+0000","last_undegraded":"2024-09-11T08:30:14.061119+0000","last_fullsized":"2024-09-11T08:30:14.061119+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:23.825873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250139+0000","last_change":"2024-09-11T08:30:05.973838+0000","last_active":"2024-09-11T08:30:14.250139+0000","last_peered":"2024-09-11T08:30:14.250139+0000","last_clean":"2024-09-11T08:30:14.250139+0000","last_became_active":"2024-09-11T08:30:05.973510+0000","last_became_peered":"2024-09-11T08:30:05.973510+0000","last_unstale":"2024-09-11T08:30:14.250139+0000","last_undegraded":"2024-09-11T08:30:14.250139+0000","last_fullsized":"2024-09-11T08:30:14.250139+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:23.872667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264331+0000","last_change":"2024-09-11T08:30:07.100331+0000","last_active":"2024-09-11T08:31:21.264331+0000","last_peered":"2024-09-11T08:31:21.264331+0000","last_clean":"2024-09-11T08:31:21.264331+0000","last_became_active":"2024-09-11T08:30:07.098073+0000","last_became_peered":"2024-09-11T08:30:07.098073+0000","last_unstale":"2024-09-11T08:31:21.264331+0000","last_undegraded":"2024-09-11T08:31:21.264331+0000","last_fullsized":"2024-09-11T08:31:21.264331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:52.832470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060798+0000","last_change":"2024-09-11T08:30:05.082285+0000","last_active":"2024-09-11T08:30:14.060798+0000","last_peered":"2024-09-11T08:30:14.060798+0000","last_clean":"2024-09-11T08:30:14.060798+0000","last_became_active":"2024-09-11T08:30:05.081895+0000","last_became_peered":"2024-09-11T08:30:05.081895+0000","last_unstale":"2024-09-11T08:30:14.060798+0000","last_undegraded":"2024-09-11T08:30:14.060798+0000","last_fullsized":"2024-09-11T08:30:14.060798+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:41.019175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678588+0000","last_change":"2024-09-11T08:30:02.991948+0000","last_active":"2024-09-11T08:30:14.678588+0000","last_peered":"2024-09-11T08:30:14.678588+0000","last_clean":"2024-09-11T08:30:14.678588+0000","last_became_active":"2024-09-11T08:30:02.991704+0000","last_became_peered":"2024-09-11T08:30:02.991704+0000","last_unstale":"2024-09-11T08:30:14.678588+0000","last_undegraded":"2024-09-11T08:30:14.678588+0000","last_fullsized":"2024-09-11T08:30:14.678588+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:15.657730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264330+0000","last_change":"2024-09-11T08:30:07.098303+0000","last_active":"2024-09-11T08:31:21.264330+0000","last_peered":"2024-09-11T08:31:21.264330+0000","last_clean":"2024-09-11T08:31:21.264330+0000","last_became_active":"2024-09-11T08:30:07.097999+0000","last_became_peered":"2024-09-11T08:30:07.097999+0000","last_unstale":"2024-09-11T08:31:21.264330+0000","last_undegraded":"2024-09-11T08:31:21.264330+0000","last_fullsized":"2024-09-11T08:31:21.264330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:28.511801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679588+0000","last_change":"2024-09-11T08:30:09.112526+0000","last_active":"2024-09-11T08:30:14.679588+0000","last_peered":"2024-09-11T08:30:14.679588+0000","last_clean":"2024-09-11T08:30:14.679588+0000","last_became_active":"2024-09-11T08:30:09.112313+0000","last_became_peered":"2024-09-11T08:30:09.112313+0000","last_unstale":"2024-09-11T08:30:14.679588+0000","last_undegraded":"2024-09-11T08:30:14.679588+0000","last_fullsized":"2024-09-11T08:30:14.679588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:38:41.753218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678254+0000","last_change":"2024-09-11T08:30:05.975282+0000","last_active":"2024-09-11T08:30:14.678254+0000","last_peered":"2024-09-11T08:30:14.678254+0000","last_clean":"2024-09-11T08:30:14.678254+0000","last_became_active":"2024-09-11T08:30:05.974834+0000","last_became_peered":"2024-09-11T08:30:05.974834+0000","last_unstale":"2024-09-11T08:30:14.678254+0000","last_undegraded":"2024-09-11T08:30:14.678254+0000","last_fullsized":"2024-09-11T08:30:14.678254+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:26.693828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679383+0000","last_change":"2024-09-11T08:30:02.997878+0000","last_active":"2024-09-11T08:30:14.679383+0000","last_peered":"2024-09-11T08:30:14.679383+0000","last_clean":"2024-09-11T08:30:14.679383+0000","last_became_active":"2024-09-11T08:30:02.997570+0000","last_became_peered":"2024-09-11T08:30:02.997570+0000","last_unstale":"2024-09-11T08:30:14.679383+0000","last_undegraded":"2024-09-11T08:30:14.679383+0000","last_fullsized":"2024-09-11T08:30:14.679383+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:43:07.844508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058712+0000","last_change":"2024-09-11T08:30:07.095053+0000","last_active":"2024-09-11T08:30:14.058712+0000","last_peered":"2024-09-11T08:30:14.058712+0000","last_clean":"2024-09-11T08:30:14.058712+0000","last_became_active":"2024-09-11T08:30:07.093412+0000","last_became_peered":"2024-09-11T08:30:07.093412+0000","last_unstale":"2024-09-11T08:30:14.058712+0000","last_undegraded":"2024-09-11T08:30:14.058712+0000","last_fullsized":"2024-09-11T08:30:14.058712+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:19.379515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058934+0000","last_change":"2024-09-11T08:30:09.108342+0000","last_active":"2024-09-11T08:30:14.058934+0000","last_peered":"2024-09-11T08:30:14.058934+0000","last_clean":"2024-09-11T08:30:14.058934+0000","last_became_active":"2024-09-11T08:30:09.108078+0000","last_became_peered":"2024-09-11T08:30:09.108078+0000","last_unstale":"2024-09-11T08:30:14.058934+0000","last_undegraded":"2024-09-11T08:30:14.058934+0000","last_fullsized":"2024-09-11T08:30:14.058934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:51:15.578994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058765+0000","last_change":"2024-09-11T08:30:05.068876+0000","last_active":"2024-09-11T08:30:14.058765+0000","last_peered":"2024-09-11T08:30:14.058765+0000","last_clean":"2024-09-11T08:30:14.058765+0000","last_became_active":"2024-09-11T08:30:05.068383+0000","last_became_peered":"2024-09-11T08:30:05.068383+0000","last_unstale":"2024-09-11T08:30:14.058765+0000","last_undegraded":"2024-09-11T08:30:14.058765+0000","last_fullsized":"2024-09-11T08:30:14.058765+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:04.854359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250331+0000","last_change":"2024-09-11T08:30:02.995890+0000","last_active":"2024-09-11T08:30:14.250331+0000","last_peered":"2024-09-11T08:30:14.250331+0000","last_clean":"2024-09-11T08:30:14.250331+0000","last_became_active":"2024-09-11T08:30:02.995557+0000","last_became_peered":"2024-09-11T08:30:02.995557+0000","last_unstale":"2024-09-11T08:30:14.250331+0000","last_undegraded":"2024-09-11T08:30:14.250331+0000","last_fullsized":"2024-09-11T08:30:14.250331+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:03.407212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264275+0000","last_change":"2024-09-11T08:30:07.094535+0000","last_active":"2024-09-11T08:31:21.264275+0000","last_peered":"2024-09-11T08:31:21.264275+0000","last_clean":"2024-09-11T08:31:21.264275+0000","last_became_active":"2024-09-11T08:30:07.094220+0000","last_became_peered":"2024-09-11T08:30:07.094220+0000","last_unstale":"2024-09-11T08:31:21.264275+0000","last_undegraded":"2024-09-11T08:31:21.264275+0000","last_fullsized":"2024-09-11T08:31:21.264275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:12:09.986404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678477+0000","last_change":"2024-09-11T08:30:09.643340+0000","last_active":"2024-09-11T08:30:14.678477+0000","last_peered":"2024-09-11T08:30:14.678477+0000","last_clean":"2024-09-11T08:30:14.678477+0000","last_became_active":"2024-09-11T08:30:09.643060+0000","last_became_peered":"2024-09-11T08:30:09.643060+0000","last_unstale":"2024-09-11T08:30:14.678477+0000","last_undegraded":"2024-09-11T08:30:14.678477+0000","last_fullsized":"2024-09-11T08:30:14.678477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:13.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059928+0000","last_change":"2024-09-11T08:30:05.073081+0000","last_active":"2024-09-11T08:30:14.059928+0000","last_peered":"2024-09-11T08:30:14.059928+0000","last_clean":"2024-09-11T08:30:14.059928+0000","last_became_active":"2024-09-11T08:30:05.072793+0000","last_became_peered":"2024-09-11T08:30:05.072793+0000","last_unstale":"2024-09-11T08:30:14.059928+0000","last_undegraded":"2024-09-11T08:30:14.059928+0000","last_fullsized":"2024-09-11T08:30:14.059928+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:07:45.710385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679896+0000","last_change":"2024-09-11T08:30:02.991940+0000","last_active":"2024-09-11T08:30:14.679896+0000","last_peered":"2024-09-11T08:30:14.679896+0000","last_clean":"2024-09-11T08:30:14.679896+0000","last_became_active":"2024-09-11T08:30:02.991660+0000","last_became_peered":"2024-09-11T08:30:02.991660+0000","last_unstale":"2024-09-11T08:30:14.679896+0000","last_undegraded":"2024-09-11T08:30:14.679896+0000","last_fullsized":"2024-09-11T08:30:14.679896+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:50.015166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264164+0000","last_change":"2024-09-11T08:30:07.103729+0000","last_active":"2024-09-11T08:31:21.264164+0000","last_peered":"2024-09-11T08:31:21.264164+0000","last_clean":"2024-09-11T08:31:21.264164+0000","last_became_active":"2024-09-11T08:30:07.103465+0000","last_became_peered":"2024-09-11T08:30:07.103465+0000","last_unstale":"2024-09-11T08:31:21.264164+0000","last_undegraded":"2024-09-11T08:31:21.264164+0000","last_fullsized":"2024-09-11T08:31:21.264164+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:23.248934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249590+0000","last_change":"2024-09-11T08:30:09.112918+0000","last_active":"2024-09-11T08:30:14.249590+0000","last_peered":"2024-09-11T08:30:14.249590+0000","last_clean":"2024-09-11T08:30:14.249590+0000","last_became_active":"2024-09-11T08:30:09.112569+0000","last_became_peered":"2024-09-11T08:30:09.112569+0000","last_unstale":"2024-09-11T08:30:14.249590+0000","last_undegraded":"2024-09-11T08:30:14.249590+0000","last_fullsized":"2024-09-11T08:30:14.249590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:43:44.615629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250102+0000","last_change":"2024-09-11T08:30:05.973857+0000","last_active":"2024-09-11T08:30:14.250102+0000","last_peered":"2024-09-11T08:30:14.250102+0000","last_clean":"2024-09-11T08:30:14.250102+0000","last_became_active":"2024-09-11T08:30:05.973576+0000","last_became_peered":"2024-09-11T08:30:05.973576+0000","last_unstale":"2024-09-11T08:30:14.250102+0000","last_undegraded":"2024-09-11T08:30:14.250102+0000","last_fullsized":"2024-09-11T08:30:14.250102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:56:58.275016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.061304+0000","last_change":"2024-09-11T08:30:02.990920+0000","last_active":"2024-09-11T08:30:14.061304+0000","last_peered":"2024-09-11T08:30:14.061304+0000","last_clean":"2024-09-11T08:30:14.061304+0000","last_became_active":"2024-09-11T08:30:02.990645+0000","last_became_peered":"2024-09-11T08:30:02.990645+0000","last_unstale":"2024-09-11T08:30:14.061304+0000","last_undegraded":"2024-09-11T08:30:14.061304+0000","last_fullsized":"2024-09-11T08:30:14.061304+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:57:53.449269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678235+0000","last_change":"2024-09-11T08:30:07.102639+0000","last_active":"2024-09-11T08:30:14.678235+0000","last_peered":"2024-09-11T08:30:14.678235+0000","last_clean":"2024-09-11T08:30:14.678235+0000","last_became_active":"2024-09-11T08:30:07.102326+0000","last_became_peered":"2024-09-11T08:30:07.102326+0000","last_unstale":"2024-09-11T08:30:14.678235+0000","last_undegraded":"2024-09-11T08:30:14.678235+0000","last_fullsized":"2024-09-11T08:30:14.678235+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:50:58.747353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059634+0000","last_change":"2024-09-11T08:30:09.641083+0000","last_active":"2024-09-11T08:30:14.059634+0000","last_peered":"2024-09-11T08:30:14.059634+0000","last_clean":"2024-09-11T08:30:14.059634+0000","last_became_active":"2024-09-11T08:30:09.640758+0000","last_became_peered":"2024-09-11T08:30:09.640758+0000","last_unstale":"2024-09-11T08:30:14.059634+0000","last_undegraded":"2024-09-11T08:30:14.059634+0000","last_fullsized":"2024-09-11T08:30:14.059634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:33.363406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":81,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264156+0000","last_change":"2024-09-11T08:30:05.082192+0000","last_active":"2024-09-11T08:31:21.264156+0000","last_peered":"2024-09-11T08:31:21.264156+0000","last_clean":"2024-09-11T08:31:21.264156+0000","last_became_active":"2024-09-11T08:30:05.081856+0000","last_became_peered":"2024-09-11T08:30:05.081856+0000","last_unstale":"2024-09-11T08:31:21.264156+0000","last_undegraded":"2024-09-11T08:31:21.264156+0000","last_fullsized":"2024-09-11T08:31:21.264156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:48.332041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059170+0000","last_change":"2024-09-11T08:30:02.997130+0000","last_active":"2024-09-11T08:30:14.059170+0000","last_peered":"2024-09-11T08:30:14.059170+0000","last_clean":"2024-09-11T08:30:14.059170+0000","last_became_active":"2024-09-11T08:30:02.996767+0000","last_became_peered":"2024-09-11T08:30:02.996767+0000","last_unstale":"2024-09-11T08:30:14.059170+0000","last_undegraded":"2024-09-11T08:30:14.059170+0000","last_fullsized":"2024-09-11T08:30:14.059170+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:00.156489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"57'2","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678989+0000","last_change":"2024-09-11T08:30:07.083427+0000","last_active":"2024-09-11T08:30:14.678989+0000","last_peered":"2024-09-11T08:30:14.678989+0000","last_clean":"2024-09-11T08:30:14.678989+0000","last_became_active":"2024-09-11T08:30:02.993706+0000","last_became_peered":"2024-09-11T08:30:02.993706+0000","last_unstale":"2024-09-11T08:30:14.678989+0000","last_undegraded":"2024-09-11T08:30:14.678989+0000","last_fullsized":"2024-09-11T08:30:14.678989+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:56:09.045984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038253099999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250068+0000","last_change":"2024-09-11T08:30:07.098131+0000","last_active":"2024-09-11T08:30:14.250068+0000","last_peered":"2024-09-11T08:30:14.250068+0000","last_clean":"2024-09-11T08:30:14.250068+0000","last_became_active":"2024-09-11T08:30:07.097851+0000","last_became_peered":"2024-09-11T08:30:07.097851+0000","last_unstale":"2024-09-11T08:30:14.250068+0000","last_undegraded":"2024-09-11T08:30:14.250068+0000","last_fullsized":"2024-09-11T08:30:14.250068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:55.270252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678593+0000","last_change":"2024-09-11T08:30:09.108749+0000","last_active":"2024-09-11T08:30:14.678593+0000","last_peered":"2024-09-11T08:30:14.678593+0000","last_clean":"2024-09-11T08:30:14.678593+0000","last_became_active":"2024-09-11T08:30:09.108464+0000","last_became_peered":"2024-09-11T08:30:09.108464+0000","last_unstale":"2024-09-11T08:30:14.678593+0000","last_undegraded":"2024-09-11T08:30:14.678593+0000","last_fullsized":"2024-09-11T08:30:14.678593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:13.637671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060884+0000","last_change":"2024-09-11T08:30:05.076405+0000","last_active":"2024-09-11T08:30:14.060884+0000","last_peered":"2024-09-11T08:30:14.060884+0000","last_clean":"2024-09-11T08:30:14.060884+0000","last_became_active":"2024-09-11T08:30:05.071843+0000","last_became_peered":"2024-09-11T08:30:05.071843+0000","last_unstale":"2024-09-11T08:30:14.060884+0000","last_undegraded":"2024-09-11T08:30:14.060884+0000","last_fullsized":"2024-09-11T08:30:14.060884+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:22.989769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060280+0000","last_change":"2024-09-11T08:30:02.985719+0000","last_active":"2024-09-11T08:30:14.060280+0000","last_peered":"2024-09-11T08:30:14.060280+0000","last_clean":"2024-09-11T08:30:14.060280+0000","last_became_active":"2024-09-11T08:30:02.985411+0000","last_became_peered":"2024-09-11T08:30:02.985411+0000","last_unstale":"2024-09-11T08:30:14.060280+0000","last_undegraded":"2024-09-11T08:30:14.060280+0000","last_fullsized":"2024-09-11T08:30:14.060280+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:32.481440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"55'1","reported_seq":24,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059093+0000","last_change":"2024-09-11T08:30:07.079602+0000","last_active":"2024-09-11T08:30:14.059093+0000","last_peered":"2024-09-11T08:30:14.059093+0000","last_clean":"2024-09-11T08:30:14.059093+0000","last_became_active":"2024-09-11T08:30:02.990758+0000","last_became_peered":"2024-09-11T08:30:02.990758+0000","last_unstale":"2024-09-11T08:30:14.059093+0000","last_undegraded":"2024-09-11T08:30:14.059093+0000","last_fullsized":"2024-09-11T08:30:14.059093+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:37.284174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000370121,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059913+0000","last_change":"2024-09-11T08:30:07.092239+0000","last_active":"2024-09-11T08:30:14.059913+0000","last_peered":"2024-09-11T08:30:14.059913+0000","last_clean":"2024-09-11T08:30:14.059913+0000","last_became_active":"2024-09-11T08:30:07.091956+0000","last_became_peered":"2024-09-11T08:30:07.091956+0000","last_unstale":"2024-09-11T08:30:14.059913+0000","last_undegraded":"2024-09-11T08:30:14.059913+0000","last_fullsized":"2024-09-11T08:30:14.059913+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:08:07.546081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678788+0000","last_change":"2024-09-11T08:30:09.643663+0000","last_active":"2024-09-11T08:30:14.678788+0000","last_peered":"2024-09-11T08:30:14.678788+0000","last_clean":"2024-09-11T08:30:14.678788+0000","last_became_active":"2024-09-11T08:30:09.643136+0000","last_became_peered":"2024-09-11T08:30:09.643136+0000","last_unstale":"2024-09-11T08:30:14.678788+0000","last_undegraded":"2024-09-11T08:30:14.678788+0000","last_fullsized":"2024-09-11T08:30:14.678788+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:18.580621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":34,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250222+0000","last_change":"2024-09-11T08:30:05.079482+0000","last_active":"2024-09-11T08:30:14.250222+0000","last_peered":"2024-09-11T08:30:14.250222+0000","last_clean":"2024-09-11T08:30:14.250222+0000","last_became_active":"2024-09-11T08:30:05.079159+0000","last_became_peered":"2024-09-11T08:30:05.079159+0000","last_unstale":"2024-09-11T08:30:14.250222+0000","last_undegraded":"2024-09-11T08:30:14.250222+0000","last_fullsized":"2024-09-11T08:30:14.250222+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:32:37.893619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249687+0000","last_change":"2024-09-11T08:30:02.995420+0000","last_active":"2024-09-11T08:30:14.249687+0000","last_peered":"2024-09-11T08:30:14.249687+0000","last_clean":"2024-09-11T08:30:14.249687+0000","last_became_active":"2024-09-11T08:30:02.995197+0000","last_became_peered":"2024-09-11T08:30:02.995197+0000","last_unstale":"2024-09-11T08:30:14.249687+0000","last_undegraded":"2024-09-11T08:30:14.249687+0000","last_fullsized":"2024-09-11T08:30:14.249687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:10.531670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'5","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:26.477166+0000","last_change":"2024-09-11T08:30:07.088060+0000","last_active":"2024-09-11T08:31:26.477166+0000","last_peered":"2024-09-11T08:31:26.477166+0000","last_clean":"2024-09-11T08:31:26.477166+0000","last_became_active":"2024-09-11T08:30:02.997083+0000","last_became_peered":"2024-09-11T08:30:02.997083+0000","last_unstale":"2024-09-11T08:31:26.477166+0000","last_undegraded":"2024-09-11T08:31:26.477166+0000","last_fullsized":"2024-09-11T08:31:26.477166+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:36.015109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039662399999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.680068+0000","last_change":"2024-09-11T08:30:07.096694+0000","last_active":"2024-09-11T08:30:14.680068+0000","last_peered":"2024-09-11T08:30:14.680068+0000","last_clean":"2024-09-11T08:30:14.680068+0000","last_became_active":"2024-09-11T08:30:07.096415+0000","last_became_peered":"2024-09-11T08:30:07.096415+0000","last_unstale":"2024-09-11T08:30:14.680068+0000","last_undegraded":"2024-09-11T08:30:14.680068+0000","last_fullsized":"2024-09-11T08:30:14.680068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:52.859229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060158+0000","last_change":"2024-09-11T08:30:09.103602+0000","last_active":"2024-09-11T08:30:14.060158+0000","last_peered":"2024-09-11T08:30:14.060158+0000","last_clean":"2024-09-11T08:30:14.060158+0000","last_became_active":"2024-09-11T08:30:09.103343+0000","last_became_peered":"2024-09-11T08:30:09.103343+0000","last_unstale":"2024-09-11T08:30:14.060158+0000","last_undegraded":"2024-09-11T08:30:14.060158+0000","last_fullsized":"2024-09-11T08:30:14.060158+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:55.546078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679330+0000","last_change":"2024-09-11T08:30:05.974728+0000","last_active":"2024-09-11T08:30:14.679330+0000","last_peered":"2024-09-11T08:30:14.679330+0000","last_clean":"2024-09-11T08:30:14.679330+0000","last_became_active":"2024-09-11T08:30:05.974443+0000","last_became_peered":"2024-09-11T08:30:05.974443+0000","last_unstale":"2024-09-11T08:30:14.679330+0000","last_undegraded":"2024-09-11T08:30:14.679330+0000","last_fullsized":"2024-09-11T08:30:14.679330+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:28:16.980240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678633+0000","last_change":"2024-09-11T08:30:02.990923+0000","last_active":"2024-09-11T08:30:14.678633+0000","last_peered":"2024-09-11T08:30:14.678633+0000","last_clean":"2024-09-11T08:30:14.678633+0000","last_became_active":"2024-09-11T08:30:02.990591+0000","last_became_peered":"2024-09-11T08:30:02.990591+0000","last_unstale":"2024-09-11T08:30:14.678633+0000","last_undegraded":"2024-09-11T08:30:14.678633+0000","last_fullsized":"2024-09-11T08:30:14.678633+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:54:31.959997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"62'79","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:30:44.206326+0000","last_change":"2024-09-11T08:29:50.832904+0000","last_active":"2024-09-11T08:30:44.206326+0000","last_peered":"2024-09-11T08:30:44.206326+0000","last_clean":"2024-09-11T08:30:44.206326+0000","last_became_active":"2024-09-11T08:29:50.828087+0000","last_became_peered":"2024-09-11T08:29:50.828087+0000","last_unstale":"2024-09-11T08:30:44.206326+0000","last_undegraded":"2024-09-11T08:30:44.206326+0000","last_fullsized":"2024-09-11T08:30:44.206326+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:27:53.850268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:27:53.850268+0000","last_clean_scrub_stamp":"2024-09-11T08:27:53.850268+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:48.200685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679048+0000","last_change":"2024-09-11T08:30:07.093476+0000","last_active":"2024-09-11T08:30:14.679048+0000","last_peered":"2024-09-11T08:30:14.679048+0000","last_clean":"2024-09-11T08:30:14.679048+0000","last_became_active":"2024-09-11T08:30:07.092977+0000","last_became_peered":"2024-09-11T08:30:07.092977+0000","last_unstale":"2024-09-11T08:30:14.679048+0000","last_undegraded":"2024-09-11T08:30:14.679048+0000","last_fullsized":"2024-09-11T08:30:14.679048+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:26:11.881790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679987+0000","last_change":"2024-09-11T08:30:09.105320+0000","last_active":"2024-09-11T08:30:14.679987+0000","last_peered":"2024-09-11T08:30:14.679987+0000","last_clean":"2024-09-11T08:30:14.679987+0000","last_became_active":"2024-09-11T08:30:09.105042+0000","last_became_peered":"2024-09-11T08:30:09.105042+0000","last_unstale":"2024-09-11T08:30:14.679987+0000","last_undegraded":"2024-09-11T08:30:14.679987+0000","last_fullsized":"2024-09-11T08:30:14.679987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:45.609579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059502+0000","last_change":"2024-09-11T08:30:05.070700+0000","last_active":"2024-09-11T08:30:14.059502+0000","last_peered":"2024-09-11T08:30:14.059502+0000","last_clean":"2024-09-11T08:30:14.059502+0000","last_became_active":"2024-09-11T08:30:05.070351+0000","last_became_peered":"2024-09-11T08:30:05.070351+0000","last_unstale":"2024-09-11T08:30:14.059502+0000","last_undegraded":"2024-09-11T08:30:14.059502+0000","last_fullsized":"2024-09-11T08:30:14.059502+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:22:42.588741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058957+0000","last_change":"2024-09-11T08:30:02.995286+0000","last_active":"2024-09-11T08:30:14.058957+0000","last_peered":"2024-09-11T08:30:14.058957+0000","last_clean":"2024-09-11T08:30:14.058957+0000","last_became_active":"2024-09-11T08:30:02.994970+0000","last_became_peered":"2024-09-11T08:30:02.994970+0000","last_unstale":"2024-09-11T08:30:14.058957+0000","last_undegraded":"2024-09-11T08:30:14.058957+0000","last_fullsized":"2024-09-11T08:30:14.058957+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:30.226149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679957+0000","last_change":"2024-09-11T08:30:07.096814+0000","last_active":"2024-09-11T08:30:14.679957+0000","last_peered":"2024-09-11T08:30:14.679957+0000","last_clean":"2024-09-11T08:30:14.679957+0000","last_became_active":"2024-09-11T08:30:07.096536+0000","last_became_peered":"2024-09-11T08:30:07.096536+0000","last_unstale":"2024-09-11T08:30:14.679957+0000","last_undegraded":"2024-09-11T08:30:14.679957+0000","last_fullsized":"2024-09-11T08:30:14.679957+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:26:17.734806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678825+0000","last_change":"2024-09-11T08:30:09.641050+0000","last_active":"2024-09-11T08:30:14.678825+0000","last_peered":"2024-09-11T08:30:14.678825+0000","last_clean":"2024-09-11T08:30:14.678825+0000","last_became_active":"2024-09-11T08:30:09.640647+0000","last_became_peered":"2024-09-11T08:30:09.640647+0000","last_unstale":"2024-09-11T08:30:14.678825+0000","last_undegraded":"2024-09-11T08:30:14.678825+0000","last_fullsized":"2024-09-11T08:30:14.678825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:30.559696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059412+0000","last_change":"2024-09-11T08:30:05.080399+0000","last_active":"2024-09-11T08:30:14.059412+0000","last_peered":"2024-09-11T08:30:14.059412+0000","last_clean":"2024-09-11T08:30:14.059412+0000","last_became_active":"2024-09-11T08:30:05.079961+0000","last_became_peered":"2024-09-11T08:30:05.079961+0000","last_unstale":"2024-09-11T08:30:14.059412+0000","last_undegraded":"2024-09-11T08:30:14.059412+0000","last_fullsized":"2024-09-11T08:30:14.059412+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:16.095070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678978+0000","last_change":"2024-09-11T08:30:02.992376+0000","last_active":"2024-09-11T08:30:14.678978+0000","last_peered":"2024-09-11T08:30:14.678978+0000","last_clean":"2024-09-11T08:30:14.678978+0000","last_became_active":"2024-09-11T08:30:02.992123+0000","last_became_peered":"2024-09-11T08:30:02.992123+0000","last_unstale":"2024-09-11T08:30:14.678978+0000","last_undegraded":"2024-09-11T08:30:14.678978+0000","last_fullsized":"2024-09-11T08:30:14.678978+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:44:08.843691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060243+0000","last_change":"2024-09-11T08:30:07.095086+0000","last_active":"2024-09-11T08:30:14.060243+0000","last_peered":"2024-09-11T08:30:14.060243+0000","last_clean":"2024-09-11T08:30:14.060243+0000","last_became_active":"2024-09-11T08:30:07.094509+0000","last_became_peered":"2024-09-11T08:30:07.094509+0000","last_unstale":"2024-09-11T08:30:14.060243+0000","last_undegraded":"2024-09-11T08:30:14.060243+0000","last_fullsized":"2024-09-11T08:30:14.060243+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:54:50.308666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060085+0000","last_change":"2024-09-11T08:30:09.109370+0000","last_active":"2024-09-11T08:30:14.060085+0000","last_peered":"2024-09-11T08:30:14.060085+0000","last_clean":"2024-09-11T08:30:14.060085+0000","last_became_active":"2024-09-11T08:30:09.109062+0000","last_became_peered":"2024-09-11T08:30:09.109062+0000","last_unstale":"2024-09-11T08:30:14.060085+0000","last_undegraded":"2024-09-11T08:30:14.060085+0000","last_fullsized":"2024-09-11T08:30:14.060085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:39.560907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059180+0000","last_change":"2024-09-11T08:30:05.073973+0000","last_active":"2024-09-11T08:30:14.059180+0000","last_peered":"2024-09-11T08:30:14.059180+0000","last_clean":"2024-09-11T08:30:14.059180+0000","last_became_active":"2024-09-11T08:30:05.073629+0000","last_became_peered":"2024-09-11T08:30:05.073629+0000","last_unstale":"2024-09-11T08:30:14.059180+0000","last_undegraded":"2024-09-11T08:30:14.059180+0000","last_fullsized":"2024-09-11T08:30:14.059180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:12.105558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678921+0000","last_change":"2024-09-11T08:30:02.991778+0000","last_active":"2024-09-11T08:30:14.678921+0000","last_peered":"2024-09-11T08:30:14.678921+0000","last_clean":"2024-09-11T08:30:14.678921+0000","last_became_active":"2024-09-11T08:30:02.991488+0000","last_became_peered":"2024-09-11T08:30:02.991488+0000","last_unstale":"2024-09-11T08:30:14.678921+0000","last_undegraded":"2024-09-11T08:30:14.678921+0000","last_fullsized":"2024-09-11T08:30:14.678921+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:52.278404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264156+0000","last_change":"2024-09-11T08:30:07.090459+0000","last_active":"2024-09-11T08:31:21.264156+0000","last_peered":"2024-09-11T08:31:21.264156+0000","last_clean":"2024-09-11T08:31:21.264156+0000","last_became_active":"2024-09-11T08:30:07.089969+0000","last_became_peered":"2024-09-11T08:30:07.089969+0000","last_unstale":"2024-09-11T08:31:21.264156+0000","last_undegraded":"2024-09-11T08:31:21.264156+0000","last_fullsized":"2024-09-11T08:31:21.264156+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:12:26.011068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059411+0000","last_change":"2024-09-11T08:30:09.109263+0000","last_active":"2024-09-11T08:30:14.059411+0000","last_peered":"2024-09-11T08:30:14.059411+0000","last_clean":"2024-09-11T08:30:14.059411+0000","last_became_active":"2024-09-11T08:30:09.108796+0000","last_became_peered":"2024-09-11T08:30:09.108796+0000","last_unstale":"2024-09-11T08:30:14.059411+0000","last_undegraded":"2024-09-11T08:30:14.059411+0000","last_fullsized":"2024-09-11T08:30:14.059411+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:13.881316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249950+0000","last_change":"2024-09-11T08:30:05.079449+0000","last_active":"2024-09-11T08:30:14.249950+0000","last_peered":"2024-09-11T08:30:14.249950+0000","last_clean":"2024-09-11T08:30:14.249950+0000","last_became_active":"2024-09-11T08:30:05.079106+0000","last_became_peered":"2024-09-11T08:30:05.079106+0000","last_unstale":"2024-09-11T08:30:14.249950+0000","last_undegraded":"2024-09-11T08:30:14.249950+0000","last_fullsized":"2024-09-11T08:30:14.249950+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:26.510253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679614+0000","last_change":"2024-09-11T08:30:02.992069+0000","last_active":"2024-09-11T08:30:14.679614+0000","last_peered":"2024-09-11T08:30:14.679614+0000","last_clean":"2024-09-11T08:30:14.679614+0000","last_became_active":"2024-09-11T08:30:02.991826+0000","last_became_peered":"2024-09-11T08:30:02.991826+0000","last_unstale":"2024-09-11T08:30:14.679614+0000","last_undegraded":"2024-09-11T08:30:14.679614+0000","last_fullsized":"2024-09-11T08:30:14.679614+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:45.876244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059050+0000","last_change":"2024-09-11T08:30:07.093358+0000","last_active":"2024-09-11T08:30:14.059050+0000","last_peered":"2024-09-11T08:30:14.059050+0000","last_clean":"2024-09-11T08:30:14.059050+0000","last_became_active":"2024-09-11T08:30:07.093088+0000","last_became_peered":"2024-09-11T08:30:07.093088+0000","last_unstale":"2024-09-11T08:30:14.059050+0000","last_undegraded":"2024-09-11T08:30:14.059050+0000","last_fullsized":"2024-09-11T08:30:14.059050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:43.377441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249669+0000","last_change":"2024-09-11T08:30:09.115039+0000","last_active":"2024-09-11T08:30:14.249669+0000","last_peered":"2024-09-11T08:30:14.249669+0000","last_clean":"2024-09-11T08:30:14.249669+0000","last_became_active":"2024-09-11T08:30:09.114745+0000","last_became_peered":"2024-09-11T08:30:09.114745+0000","last_unstale":"2024-09-11T08:30:14.249669+0000","last_undegraded":"2024-09-11T08:30:14.249669+0000","last_fullsized":"2024-09-11T08:30:14.249669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:54:08.962931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678793+0000","last_change":"2024-09-11T08:30:05.974089+0000","last_active":"2024-09-11T08:30:14.678793+0000","last_peered":"2024-09-11T08:30:14.678793+0000","last_clean":"2024-09-11T08:30:14.678793+0000","last_became_active":"2024-09-11T08:30:05.973721+0000","last_became_peered":"2024-09-11T08:30:05.973721+0000","last_unstale":"2024-09-11T08:30:14.678793+0000","last_undegraded":"2024-09-11T08:30:14.678793+0000","last_fullsized":"2024-09-11T08:30:14.678793+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:03.167397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679104+0000","last_change":"2024-09-11T08:30:02.994171+0000","last_active":"2024-09-11T08:30:14.679104+0000","last_peered":"2024-09-11T08:30:14.679104+0000","last_clean":"2024-09-11T08:30:14.679104+0000","last_became_active":"2024-09-11T08:30:02.993912+0000","last_became_peered":"2024-09-11T08:30:02.993912+0000","last_unstale":"2024-09-11T08:30:14.679104+0000","last_undegraded":"2024-09-11T08:30:14.679104+0000","last_fullsized":"2024-09-11T08:30:14.679104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:33.612107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059354+0000","last_change":"2024-09-11T08:30:07.099628+0000","last_active":"2024-09-11T08:30:14.059354+0000","last_peered":"2024-09-11T08:30:14.059354+0000","last_clean":"2024-09-11T08:30:14.059354+0000","last_became_active":"2024-09-11T08:30:07.099281+0000","last_became_peered":"2024-09-11T08:30:07.099281+0000","last_unstale":"2024-09-11T08:30:14.059354+0000","last_undegraded":"2024-09-11T08:30:14.059354+0000","last_fullsized":"2024-09-11T08:30:14.059354+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:24.194742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678216+0000","last_change":"2024-09-11T08:30:09.111403+0000","last_active":"2024-09-11T08:30:14.678216+0000","last_peered":"2024-09-11T08:30:14.678216+0000","last_clean":"2024-09-11T08:30:14.678216+0000","last_became_active":"2024-09-11T08:30:09.111094+0000","last_became_peered":"2024-09-11T08:30:09.111094+0000","last_unstale":"2024-09-11T08:30:14.678216+0000","last_undegraded":"2024-09-11T08:30:14.678216+0000","last_fullsized":"2024-09-11T08:30:14.678216+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:40.889055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058869+0000","last_change":"2024-09-11T08:30:05.075489+0000","last_active":"2024-09-11T08:30:14.058869+0000","last_peered":"2024-09-11T08:30:14.058869+0000","last_clean":"2024-09-11T08:30:14.058869+0000","last_became_active":"2024-09-11T08:30:05.075180+0000","last_became_peered":"2024-09-11T08:30:05.075180+0000","last_unstale":"2024-09-11T08:30:14.058869+0000","last_undegraded":"2024-09-11T08:30:14.058869+0000","last_fullsized":"2024-09-11T08:30:14.058869+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:35.928928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.680366+0000","last_change":"2024-09-11T08:30:02.998911+0000","last_active":"2024-09-11T08:30:14.680366+0000","last_peered":"2024-09-11T08:30:14.680366+0000","last_clean":"2024-09-11T08:30:14.680366+0000","last_became_active":"2024-09-11T08:30:02.998485+0000","last_became_peered":"2024-09-11T08:30:02.998485+0000","last_unstale":"2024-09-11T08:30:14.680366+0000","last_undegraded":"2024-09-11T08:30:14.680366+0000","last_fullsized":"2024-09-11T08:30:14.680366+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:53.343966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.058875+0000","last_change":"2024-09-11T08:30:07.100401+0000","last_active":"2024-09-11T08:30:14.058875+0000","last_peered":"2024-09-11T08:30:14.058875+0000","last_clean":"2024-09-11T08:30:14.058875+0000","last_became_active":"2024-09-11T08:30:07.099908+0000","last_became_peered":"2024-09-11T08:30:07.099908+0000","last_unstale":"2024-09-11T08:30:14.058875+0000","last_undegraded":"2024-09-11T08:30:14.058875+0000","last_fullsized":"2024-09-11T08:30:14.058875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:44:36.931005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678585+0000","last_change":"2024-09-11T08:30:09.108063+0000","last_active":"2024-09-11T08:30:14.678585+0000","last_peered":"2024-09-11T08:30:14.678585+0000","last_clean":"2024-09-11T08:30:14.678585+0000","last_became_active":"2024-09-11T08:30:09.107722+0000","last_became_peered":"2024-09-11T08:30:09.107722+0000","last_unstale":"2024-09-11T08:30:14.678585+0000","last_undegraded":"2024-09-11T08:30:14.678585+0000","last_fullsized":"2024-09-11T08:30:14.678585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:35.304732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679425+0000","last_change":"2024-09-11T08:30:05.974622+0000","last_active":"2024-09-11T08:30:14.679425+0000","last_peered":"2024-09-11T08:30:14.679425+0000","last_clean":"2024-09-11T08:30:14.679425+0000","last_became_active":"2024-09-11T08:30:05.974284+0000","last_became_peered":"2024-09-11T08:30:05.974284+0000","last_unstale":"2024-09-11T08:30:14.679425+0000","last_undegraded":"2024-09-11T08:30:14.679425+0000","last_fullsized":"2024-09-11T08:30:14.679425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:39.981411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'2","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.680517+0000","last_change":"2024-09-11T08:30:02.998437+0000","last_active":"2024-09-11T08:30:14.680517+0000","last_peered":"2024-09-11T08:30:14.680517+0000","last_clean":"2024-09-11T08:30:14.680517+0000","last_became_active":"2024-09-11T08:30:02.998098+0000","last_became_peered":"2024-09-11T08:30:02.998098+0000","last_unstale":"2024-09-11T08:30:14.680517+0000","last_undegraded":"2024-09-11T08:30:14.680517+0000","last_fullsized":"2024-09-11T08:30:14.680517+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:28.205808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264331+0000","last_change":"2024-09-11T08:30:07.096855+0000","last_active":"2024-09-11T08:31:21.264331+0000","last_peered":"2024-09-11T08:31:21.264331+0000","last_clean":"2024-09-11T08:31:21.264331+0000","last_became_active":"2024-09-11T08:30:07.096521+0000","last_became_peered":"2024-09-11T08:30:07.096521+0000","last_unstale":"2024-09-11T08:31:21.264331+0000","last_undegraded":"2024-09-11T08:31:21.264331+0000","last_fullsized":"2024-09-11T08:31:21.264331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:56:50.703866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678609+0000","last_change":"2024-09-11T08:30:09.640736+0000","last_active":"2024-09-11T08:30:14.678609+0000","last_peered":"2024-09-11T08:30:14.678609+0000","last_clean":"2024-09-11T08:30:14.678609+0000","last_became_active":"2024-09-11T08:30:09.640413+0000","last_became_peered":"2024-09-11T08:30:09.640413+0000","last_unstale":"2024-09-11T08:30:14.678609+0000","last_undegraded":"2024-09-11T08:30:14.678609+0000","last_fullsized":"2024-09-11T08:30:14.678609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:11:26.409559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678590+0000","last_change":"2024-09-11T08:30:02.997909+0000","last_active":"2024-09-11T08:30:14.678590+0000","last_peered":"2024-09-11T08:30:14.678590+0000","last_clean":"2024-09-11T08:30:14.678590+0000","last_became_active":"2024-09-11T08:30:02.997606+0000","last_became_peered":"2024-09-11T08:30:02.997606+0000","last_unstale":"2024-09-11T08:30:14.678590+0000","last_undegraded":"2024-09-11T08:30:14.678590+0000","last_fullsized":"2024-09-11T08:30:14.678590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:25.397503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679319+0000","last_change":"2024-09-11T08:30:05.974206+0000","last_active":"2024-09-11T08:30:14.679319+0000","last_peered":"2024-09-11T08:30:14.679319+0000","last_clean":"2024-09-11T08:30:14.679319+0000","last_became_active":"2024-09-11T08:30:05.973567+0000","last_became_peered":"2024-09-11T08:30:05.973567+0000","last_unstale":"2024-09-11T08:30:14.679319+0000","last_undegraded":"2024-09-11T08:30:14.679319+0000","last_fullsized":"2024-09-11T08:30:14.679319+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:26.872664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679797+0000","last_change":"2024-09-11T08:30:07.091157+0000","last_active":"2024-09-11T08:30:14.679797+0000","last_peered":"2024-09-11T08:30:14.679797+0000","last_clean":"2024-09-11T08:30:14.679797+0000","last_became_active":"2024-09-11T08:30:07.090852+0000","last_became_peered":"2024-09-11T08:30:07.090852+0000","last_unstale":"2024-09-11T08:30:14.679797+0000","last_undegraded":"2024-09-11T08:30:14.679797+0000","last_fullsized":"2024-09-11T08:30:14.679797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:03.448113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678698+0000","last_change":"2024-09-11T08:30:09.641099+0000","last_active":"2024-09-11T08:30:14.678698+0000","last_peered":"2024-09-11T08:30:14.678698+0000","last_clean":"2024-09-11T08:30:14.678698+0000","last_became_active":"2024-09-11T08:30:09.640806+0000","last_became_peered":"2024-09-11T08:30:09.640806+0000","last_unstale":"2024-09-11T08:30:14.678698+0000","last_undegraded":"2024-09-11T08:30:14.678698+0000","last_fullsized":"2024-09-11T08:30:14.678698+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:04:22.496535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250267+0000","last_change":"2024-09-11T08:30:05.975575+0000","last_active":"2024-09-11T08:30:14.250267+0000","last_peered":"2024-09-11T08:30:14.250267+0000","last_clean":"2024-09-11T08:30:14.250267+0000","last_became_active":"2024-09-11T08:30:05.974386+0000","last_became_peered":"2024-09-11T08:30:05.974386+0000","last_unstale":"2024-09-11T08:30:14.250267+0000","last_undegraded":"2024-09-11T08:30:14.250267+0000","last_fullsized":"2024-09-11T08:30:14.250267+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:53.649662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678611+0000","last_change":"2024-09-11T08:30:02.999621+0000","last_active":"2024-09-11T08:30:14.678611+0000","last_peered":"2024-09-11T08:30:14.678611+0000","last_clean":"2024-09-11T08:30:14.678611+0000","last_became_active":"2024-09-11T08:30:02.998649+0000","last_became_peered":"2024-09-11T08:30:02.998649+0000","last_unstale":"2024-09-11T08:30:14.678611+0000","last_undegraded":"2024-09-11T08:30:14.678611+0000","last_fullsized":"2024-09-11T08:30:14.678611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:17:38.749919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679178+0000","last_change":"2024-09-11T08:30:07.093621+0000","last_active":"2024-09-11T08:30:14.679178+0000","last_peered":"2024-09-11T08:30:14.679178+0000","last_clean":"2024-09-11T08:30:14.679178+0000","last_became_active":"2024-09-11T08:30:07.092092+0000","last_became_peered":"2024-09-11T08:30:07.092092+0000","last_unstale":"2024-09-11T08:30:14.679178+0000","last_undegraded":"2024-09-11T08:30:14.679178+0000","last_fullsized":"2024-09-11T08:30:14.679178+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:31.169507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059733+0000","last_change":"2024-09-11T08:30:09.110871+0000","last_active":"2024-09-11T08:30:14.059733+0000","last_peered":"2024-09-11T08:30:14.059733+0000","last_clean":"2024-09-11T08:30:14.059733+0000","last_became_active":"2024-09-11T08:30:09.110548+0000","last_became_peered":"2024-09-11T08:30:09.110548+0000","last_unstale":"2024-09-11T08:30:14.059733+0000","last_undegraded":"2024-09-11T08:30:14.059733+0000","last_fullsized":"2024-09-11T08:30:14.059733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:08:01.967506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679795+0000","last_change":"2024-09-11T08:30:05.974817+0000","last_active":"2024-09-11T08:30:14.679795+0000","last_peered":"2024-09-11T08:30:14.679795+0000","last_clean":"2024-09-11T08:30:14.679795+0000","last_became_active":"2024-09-11T08:30:05.974399+0000","last_became_peered":"2024-09-11T08:30:05.974399+0000","last_unstale":"2024-09-11T08:30:14.679795+0000","last_undegraded":"2024-09-11T08:30:14.679795+0000","last_fullsized":"2024-09-11T08:30:14.679795+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:10.562163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249594+0000","last_change":"2024-09-11T08:30:02.992432+0000","last_active":"2024-09-11T08:30:14.249594+0000","last_peered":"2024-09-11T08:30:14.249594+0000","last_clean":"2024-09-11T08:30:14.249594+0000","last_became_active":"2024-09-11T08:30:02.992145+0000","last_became_peered":"2024-09-11T08:30:02.992145+0000","last_unstale":"2024-09-11T08:30:14.249594+0000","last_undegraded":"2024-09-11T08:30:14.249594+0000","last_fullsized":"2024-09-11T08:30:14.249594+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:42:14.459417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264274+0000","last_change":"2024-09-11T08:30:07.094423+0000","last_active":"2024-09-11T08:31:21.264274+0000","last_peered":"2024-09-11T08:31:21.264274+0000","last_clean":"2024-09-11T08:31:21.264274+0000","last_became_active":"2024-09-11T08:30:07.094081+0000","last_became_peered":"2024-09-11T08:30:07.094081+0000","last_unstale":"2024-09-11T08:31:21.264274+0000","last_undegraded":"2024-09-11T08:31:21.264274+0000","last_fullsized":"2024-09-11T08:31:21.264274+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:06:36.885569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059548+0000","last_change":"2024-09-11T08:30:09.111164+0000","last_active":"2024-09-11T08:30:14.059548+0000","last_peered":"2024-09-11T08:30:14.059548+0000","last_clean":"2024-09-11T08:30:14.059548+0000","last_became_active":"2024-09-11T08:30:09.110662+0000","last_became_peered":"2024-09-11T08:30:09.110662+0000","last_unstale":"2024-09-11T08:30:14.059548+0000","last_undegraded":"2024-09-11T08:30:14.059548+0000","last_fullsized":"2024-09-11T08:30:14.059548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:35.329576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679215+0000","last_change":"2024-09-11T08:30:05.973885+0000","last_active":"2024-09-11T08:30:14.679215+0000","last_peered":"2024-09-11T08:30:14.679215+0000","last_clean":"2024-09-11T08:30:14.679215+0000","last_became_active":"2024-09-11T08:30:05.973490+0000","last_became_peered":"2024-09-11T08:30:05.973490+0000","last_unstale":"2024-09-11T08:30:14.679215+0000","last_undegraded":"2024-09-11T08:30:14.679215+0000","last_fullsized":"2024-09-11T08:30:14.679215+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:08:05.884036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678773+0000","last_change":"2024-09-11T08:30:02.996121+0000","last_active":"2024-09-11T08:30:14.678773+0000","last_peered":"2024-09-11T08:30:14.678773+0000","last_clean":"2024-09-11T08:30:14.678773+0000","last_became_active":"2024-09-11T08:30:02.995828+0000","last_became_peered":"2024-09-11T08:30:02.995828+0000","last_unstale":"2024-09-11T08:30:14.678773+0000","last_undegraded":"2024-09-11T08:30:14.678773+0000","last_fullsized":"2024-09-11T08:30:14.678773+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:05.529343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T08:31:21.264329+0000","last_change":"2024-09-11T08:30:07.100063+0000","last_active":"2024-09-11T08:31:21.264329+0000","last_peered":"2024-09-11T08:31:21.264329+0000","last_clean":"2024-09-11T08:31:21.264329+0000","last_became_active":"2024-09-11T08:30:07.099752+0000","last_became_peered":"2024-09-11T08:30:07.099752+0000","last_unstale":"2024-09-11T08:31:21.264329+0000","last_undegraded":"2024-09-11T08:31:21.264329+0000","last_fullsized":"2024-09-11T08:31:21.264329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:04:19.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250017+0000","last_change":"2024-09-11T08:30:09.114755+0000","last_active":"2024-09-11T08:30:14.250017+0000","last_peered":"2024-09-11T08:30:14.250017+0000","last_clean":"2024-09-11T08:30:14.250017+0000","last_became_active":"2024-09-11T08:30:09.114428+0000","last_became_peered":"2024-09-11T08:30:09.114428+0000","last_unstale":"2024-09-11T08:30:14.250017+0000","last_undegraded":"2024-09-11T08:30:14.250017+0000","last_fullsized":"2024-09-11T08:30:14.250017+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:21.459812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059036+0000","last_change":"2024-09-11T08:30:05.080269+0000","last_active":"2024-09-11T08:30:14.059036+0000","last_peered":"2024-09-11T08:30:14.059036+0000","last_clean":"2024-09-11T08:30:14.059036+0000","last_became_active":"2024-09-11T08:30:05.079754+0000","last_became_peered":"2024-09-11T08:30:05.079754+0000","last_unstale":"2024-09-11T08:30:14.059036+0000","last_undegraded":"2024-09-11T08:30:14.059036+0000","last_fullsized":"2024-09-11T08:30:14.059036+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:27.996903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"53'1","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059266+0000","last_change":"2024-09-11T08:30:02.996409+0000","last_active":"2024-09-11T08:30:14.059266+0000","last_peered":"2024-09-11T08:30:14.059266+0000","last_clean":"2024-09-11T08:30:14.059266+0000","last_became_active":"2024-09-11T08:30:02.996127+0000","last_became_peered":"2024-09-11T08:30:02.996127+0000","last_unstale":"2024-09-11T08:30:14.059266+0000","last_undegraded":"2024-09-11T08:30:14.059266+0000","last_fullsized":"2024-09-11T08:30:14.059266+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:12.139498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.059742+0000","last_change":"2024-09-11T08:30:07.090330+0000","last_active":"2024-09-11T08:30:14.059742+0000","last_peered":"2024-09-11T08:30:14.059742+0000","last_clean":"2024-09-11T08:30:14.059742+0000","last_became_active":"2024-09-11T08:30:07.088941+0000","last_became_peered":"2024-09-11T08:30:07.088941+0000","last_unstale":"2024-09-11T08:30:14.059742+0000","last_undegraded":"2024-09-11T08:30:14.059742+0000","last_fullsized":"2024-09-11T08:30:14.059742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:24.899987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678194+0000","last_change":"2024-09-11T08:30:09.113876+0000","last_active":"2024-09-11T08:30:14.678194+0000","last_peered":"2024-09-11T08:30:14.678194+0000","last_clean":"2024-09-11T08:30:14.678194+0000","last_became_active":"2024-09-11T08:30:09.113500+0000","last_became_peered":"2024-09-11T08:30:09.113500+0000","last_unstale":"2024-09-11T08:30:14.678194+0000","last_undegraded":"2024-09-11T08:30:14.678194+0000","last_fullsized":"2024-09-11T08:30:14.678194+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:47.202797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060024+0000","last_change":"2024-09-11T08:30:05.070804+0000","last_active":"2024-09-11T08:30:14.060024+0000","last_peered":"2024-09-11T08:30:14.060024+0000","last_clean":"2024-09-11T08:30:14.060024+0000","last_became_active":"2024-09-11T08:30:05.070470+0000","last_became_peered":"2024-09-11T08:30:05.070470+0000","last_unstale":"2024-09-11T08:30:14.060024+0000","last_undegraded":"2024-09-11T08:30:14.060024+0000","last_fullsized":"2024-09-11T08:30:14.060024+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:35.006562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678979+0000","last_change":"2024-09-11T08:30:02.996378+0000","last_active":"2024-09-11T08:30:14.678979+0000","last_peered":"2024-09-11T08:30:14.678979+0000","last_clean":"2024-09-11T08:30:14.678979+0000","last_became_active":"2024-09-11T08:30:02.996118+0000","last_became_peered":"2024-09-11T08:30:02.996118+0000","last_unstale":"2024-09-11T08:30:14.678979+0000","last_undegraded":"2024-09-11T08:30:14.678979+0000","last_fullsized":"2024-09-11T08:30:14.678979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:34:39.261710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678418+0000","last_change":"2024-09-11T08:30:07.098119+0000","last_active":"2024-09-11T08:30:14.678418+0000","last_peered":"2024-09-11T08:30:14.678418+0000","last_clean":"2024-09-11T08:30:14.678418+0000","last_became_active":"2024-09-11T08:30:07.097776+0000","last_became_peered":"2024-09-11T08:30:07.097776+0000","last_unstale":"2024-09-11T08:30:14.678418+0000","last_undegraded":"2024-09-11T08:30:14.678418+0000","last_fullsized":"2024-09-11T08:30:14.678418+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:54.678730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.249324+0000","last_change":"2024-09-11T08:30:09.112757+0000","last_active":"2024-09-11T08:30:14.249324+0000","last_peered":"2024-09-11T08:30:14.249324+0000","last_clean":"2024-09-11T08:30:14.249324+0000","last_became_active":"2024-09-11T08:30:09.112410+0000","last_became_peered":"2024-09-11T08:30:09.112410+0000","last_unstale":"2024-09-11T08:30:14.249324+0000","last_undegraded":"2024-09-11T08:30:14.249324+0000","last_fullsized":"2024-09-11T08:30:14.249324+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:56.666041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678358+0000","last_change":"2024-09-11T08:30:05.974060+0000","last_active":"2024-09-11T08:30:14.678358+0000","last_peered":"2024-09-11T08:30:14.678358+0000","last_clean":"2024-09-11T08:30:14.678358+0000","last_became_active":"2024-09-11T08:30:05.973805+0000","last_became_peered":"2024-09-11T08:30:05.973805+0000","last_unstale":"2024-09-11T08:30:14.678358+0000","last_undegraded":"2024-09-11T08:30:14.678358+0000","last_fullsized":"2024-09-11T08:30:14.678358+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:20:54.951937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679846+0000","last_change":"2024-09-11T08:30:02.991063+0000","last_active":"2024-09-11T08:30:14.679846+0000","last_peered":"2024-09-11T08:30:14.679846+0000","last_clean":"2024-09-11T08:30:14.679846+0000","last_became_active":"2024-09-11T08:30:02.990625+0000","last_became_peered":"2024-09-11T08:30:02.990625+0000","last_unstale":"2024-09-11T08:30:14.679846+0000","last_undegraded":"2024-09-11T08:30:14.679846+0000","last_fullsized":"2024-09-11T08:30:14.679846+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:07:12.143627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678036+0000","last_change":"2024-09-11T08:30:07.094339+0000","last_active":"2024-09-11T08:30:14.678036+0000","last_peered":"2024-09-11T08:30:14.678036+0000","last_clean":"2024-09-11T08:30:14.678036+0000","last_became_active":"2024-09-11T08:30:07.094034+0000","last_became_peered":"2024-09-11T08:30:07.094034+0000","last_unstale":"2024-09-11T08:30:14.678036+0000","last_undegraded":"2024-09-11T08:30:14.678036+0000","last_fullsized":"2024-09-11T08:30:14.678036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:47.199571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678979+0000","last_change":"2024-09-11T08:30:09.643435+0000","last_active":"2024-09-11T08:30:14.678979+0000","last_peered":"2024-09-11T08:30:14.678979+0000","last_clean":"2024-09-11T08:30:14.678979+0000","last_became_active":"2024-09-11T08:30:09.643128+0000","last_became_peered":"2024-09-11T08:30:09.643128+0000","last_unstale":"2024-09-11T08:30:14.678979+0000","last_undegraded":"2024-09-11T08:30:14.678979+0000","last_fullsized":"2024-09-11T08:30:14.678979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:04.979550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678192+0000","last_change":"2024-09-11T08:30:05.080210+0000","last_active":"2024-09-11T08:30:14.678192+0000","last_peered":"2024-09-11T08:30:14.678192+0000","last_clean":"2024-09-11T08:30:14.678192+0000","last_became_active":"2024-09-11T08:30:05.079930+0000","last_became_peered":"2024-09-11T08:30:05.079930+0000","last_unstale":"2024-09-11T08:30:14.678192+0000","last_undegraded":"2024-09-11T08:30:14.678192+0000","last_fullsized":"2024-09-11T08:30:14.678192+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:54.261529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250346+0000","last_change":"2024-09-11T08:30:02.995939+0000","last_active":"2024-09-11T08:30:14.250346+0000","last_peered":"2024-09-11T08:30:14.250346+0000","last_clean":"2024-09-11T08:30:14.250346+0000","last_became_active":"2024-09-11T08:30:02.995620+0000","last_became_peered":"2024-09-11T08:30:02.995620+0000","last_unstale":"2024-09-11T08:30:14.250346+0000","last_undegraded":"2024-09-11T08:30:14.250346+0000","last_fullsized":"2024-09-11T08:30:14.250346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:46.695273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678276+0000","last_change":"2024-09-11T08:30:07.099011+0000","last_active":"2024-09-11T08:30:14.678276+0000","last_peered":"2024-09-11T08:30:14.678276+0000","last_clean":"2024-09-11T08:30:14.678276+0000","last_became_active":"2024-09-11T08:30:07.098633+0000","last_became_peered":"2024-09-11T08:30:07.098633+0000","last_unstale":"2024-09-11T08:30:14.678276+0000","last_undegraded":"2024-09-11T08:30:14.678276+0000","last_fullsized":"2024-09-11T08:30:14.678276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:06.219514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678426+0000","last_change":"2024-09-11T08:30:09.109045+0000","last_active":"2024-09-11T08:30:14.678426+0000","last_peered":"2024-09-11T08:30:14.678426+0000","last_clean":"2024-09-11T08:30:14.678426+0000","last_became_active":"2024-09-11T08:30:09.108734+0000","last_became_peered":"2024-09-11T08:30:09.108734+0000","last_unstale":"2024-09-11T08:30:14.678426+0000","last_undegraded":"2024-09-11T08:30:14.678426+0000","last_fullsized":"2024-09-11T08:30:14.678426+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:41.055857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.060031+0000","last_change":"2024-09-11T08:30:09.102625+0000","last_active":"2024-09-11T08:30:14.060031+0000","last_peered":"2024-09-11T08:30:14.060031+0000","last_clean":"2024-09-11T08:30:14.060031+0000","last_became_active":"2024-09-11T08:30:09.102309+0000","last_became_peered":"2024-09-11T08:30:09.102309+0000","last_unstale":"2024-09-11T08:30:14.060031+0000","last_undegraded":"2024-09-11T08:30:14.060031+0000","last_fullsized":"2024-09-11T08:30:14.060031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:08.085352+0000","last_clean_scrub_stamp":"2024-09-11T08:30:08.085352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:04.315237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"53'1","reported_seq":27,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.678388+0000","last_change":"2024-09-11T08:30:02.997681+0000","last_active":"2024-09-11T08:30:14.678388+0000","last_peered":"2024-09-11T08:30:14.678388+0000","last_clean":"2024-09-11T08:30:14.678388+0000","last_became_active":"2024-09-11T08:30:02.997422+0000","last_became_peered":"2024-09-11T08:30:02.997422+0000","last_unstale":"2024-09-11T08:30:14.678388+0000","last_undegraded":"2024-09-11T08:30:14.678388+0000","last_fullsized":"2024-09-11T08:30:14.678388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:01.968771+0000","last_clean_scrub_stamp":"2024-09-11T08:30:01.968771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:26.866200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.679489+0000","last_change":"2024-09-11T08:30:05.082897+0000","last_active":"2024-09-11T08:30:14.679489+0000","last_peered":"2024-09-11T08:30:14.679489+0000","last_clean":"2024-09-11T08:30:14.679489+0000","last_became_active":"2024-09-11T08:30:05.082529+0000","last_became_peered":"2024-09-11T08:30:05.082529+0000","last_unstale":"2024-09-11T08:30:14.679489+0000","last_undegraded":"2024-09-11T08:30:14.679489+0000","last_fullsized":"2024-09-11T08:30:14.679489+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:04.054731+0000","last_clean_scrub_stamp":"2024-09-11T08:30:04.054731+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:36.776919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:30:14.250045+0000","last_change":"2024-09-11T08:30:07.097926+0000","last_active":"2024-09-11T08:30:14.250045+0000","last_peered":"2024-09-11T08:30:14.250045+0000","last_clean":"2024-09-11T08:30:14.250045+0000","last_became_active":"2024-09-11T08:30:07.097600+0000","last_became_peered":"2024-09-11T08:30:07.097600+0000","last_unstale":"2024-09-11T08:30:14.250045+0000","last_undegraded":"2024-09-11T08:30:14.250045+0000","last_fullsized":"2024-09-11T08:30:14.250045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:30:06.071227+0000","last_clean_scrub_stamp":"2024-09-11T08:30:06.071227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:07:52.470385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":73,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397526,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298548,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446604,"statfs":{"total":95995035648,"available":95689322496,"internally_reserved":0,"allocated":1224704,"data_stored":825707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.56699999999999995}]},{"osd":1,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.499}]},{"osd":2,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.45000000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.621}]},{"osd":6,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.48799999999999999}]}]},{"osd":6,"up_from":43,"seq":184683593755,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298520,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446632,"statfs":{"total":95995035648,"available":95689351168,"internally_reserved":0,"allocated":1196032,"data_stored":824097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.44}]},{"osd":1,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.434,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74299999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.47099999999999997}]},{"osd":4,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.376},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.42599999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.41999999999999998,"15min":0.40200000000000002},"min":{"1min":0.32800000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.35499999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.45300000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789985,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297944,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447208,"statfs":{"total":95995035648,"available":95689940992,"internally_reserved":0,"allocated":602112,"data_stored":233923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.70999999999999996,"5min":0.87,"15min":0.87},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81699999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82699999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55000000000000004}]},{"osd":3,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41299999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.32700000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40899999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297960,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447192,"statfs":{"total":95995035648,"available":95689924608,"internally_reserved":0,"allocated":618496,"data_stored":234208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66500000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.71299999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.51800000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:31:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.29199999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.378},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.47799999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.432}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":235645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36899999999999999,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52800000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.308},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.40899999999999997,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.28799999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.41199999999999998,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.48899999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.39200000000000002,"15min":0.40200000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.38300000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.29899999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.50600000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34200000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80600000000000005,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51700000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67200000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.50900000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.52700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69099999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.69299999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:30:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.434},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48099999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":234027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35999999999999999,"15min":0.35399999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.48799999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.375},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.375,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.31}]},{"osd":1,"last update":"Wed Sep 11 08:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.26600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41799999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65100000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.28499999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46000000000000002,"15min":0.47299999999999998},"min":{"1min":0.29699999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.64200000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.48099999999999998,"15min":0.498},"min":{"1min":0.32200000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65900000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.33300000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65100000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41199999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.434},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.70399999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45800000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:30:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":663552,"data_stored":236480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.377,"15min":0.34499999999999997},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.433,"15min":0.39000000000000001},"min":{"1min":0.249,"5min":0.218,"15min":0.218},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.39200000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.443},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59199999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.31,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64400000000000002,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.31}]},{"osd":4,"last update":"Wed Sep 11 08:30:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.501},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79900000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86499999999999999,"5min":1.482,"15min":1.482},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74299999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Sep 11 08:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.63100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1220608,"data_stored":825903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.32100000000000001,"15min":0.29399999999999998},"min":{"1min":0.248,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54300000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.32700000000000001,"15min":0.30399999999999999},"min":{"1min":0.26200000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.24199999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.25600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.41299999999999998,"15min":0.39900000000000002},"min":{"1min":0.32600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:31:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.53200000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32200000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:30:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.441},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.39400000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.41899999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42599999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60099999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.36799999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:31:32.893 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T08:31:32.893 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T08:31:32.893 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T08:31:32.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 327ee848-7017-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T08:31:33.156 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:31:33.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:33 smithi172 ceph-mon[32987]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:31:33.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:33 smithi172 ceph-mon[32987]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:33.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:33 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:33.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:33 smithi043 ceph-mon[40592]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:31:33.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:33 smithi043 ceph-mon[40592]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:33.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:33 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:33.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:33 smithi043 ceph-mon[29355]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:31:33.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:33 smithi043 ceph-mon[29355]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:33.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:33 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 -- 172.21.15.43:0/1250038197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 msgr2=0x7f6e54102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 --2- 172.21.15.43:0/1250038197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 0x7f6e54102be0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e44004890 tx=0x7f6e44009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 -- 172.21.15.43:0/1250038197 shutdown_connections 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 --2- 172.21.15.43:0/1250038197 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e541031b0 0x7f6e540fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 --2- 172.21.15.43:0/1250038197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e540fb440 0x7f6e54109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 --2- 172.21.15.43:0/1250038197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 0x7f6e54102be0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 -- 172.21.15.43:0/1250038197 >> 172.21.15.43:0/1250038197 conn(0x7f6e54074c80 msgr2=0x7f6e540f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 -- 172.21.15.43:0/1250038197 shutdown_connections 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 -- 172.21.15.43:0/1250038197 wait complete. 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.940+0000 7f6e594a1640 1 Processor -- start 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e594a1640 1 -- start start 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e594a1640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e540fb440 0x7f6e54114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e594a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 0x7f6e54115450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e594a1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e541031b0 0x7f6e5410f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:34.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e594a1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5410f6f0 con 0x7f6e54102810 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e594a1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5410f860 con 0x7f6e541031b0 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e594a1640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5410fa20 con 0x7f6e540fb440 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e52ffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e540fb440 0x7f6e54114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e527fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 0x7f6e54115450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e52ffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e540fb440 0x7f6e54114f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:54308/0 (socket says 172.21.15.43:54308) 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e52ffd640 1 -- 172.21.15.43:0/191282333 learned_addr learned my addr 172.21.15.43:0/191282333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e537fe640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e541031b0 0x7f6e5410f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e52ffd640 1 -- 172.21.15.43:0/191282333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e541031b0 msgr2=0x7f6e5410f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e52ffd640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e541031b0 0x7f6e5410f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e52ffd640 1 -- 172.21.15.43:0/191282333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 msgr2=0x7f6e54115450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:34.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e52ffd640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 0x7f6e54115450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e52ffd640 1 -- 172.21.15.43:0/191282333 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48005040 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.941+0000 7f6e527fc640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 0x7f6e54115450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.942+0000 7f6e52ffd640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e540fb440 0x7f6e54114f10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e4402a4a0 tx=0x7f6e44009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.942+0000 7f6e3b7fe640 1 -- 172.21.15.43:0/191282333 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e440092e0 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.942+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e44005040 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.942+0000 7f6e3b7fe640 1 -- 172.21.15.43:0/191282333 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f6e440095a0 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.942+0000 7f6e3b7fe640 1 -- 172.21.15.43:0/191282333 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44033690 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.942+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5411bd60 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.943+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e540fe030 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.944+0000 7f6e3b7fe640 1 -- 172.21.15.43:0/191282333 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f6e44030020 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.944+0000 7f6e3b7fe640 1 --2- 172.21.15.43:0/191282333 >> v2:172.21.15.43:6800/588375333 conn(0x7f6e3c06ef80 0x7f6e3c071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.944+0000 7f6e3b7fe640 1 -- 172.21.15.43:0/191282333 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f6e440ae0a0 con 0x7f6e540fb440 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.944+0000 7f6e527fc640 1 --2- 172.21.15.43:0/191282333 >> v2:172.21.15.43:6800/588375333 conn(0x7f6e3c06ef80 0x7f6e3c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.945+0000 7f6e527fc640 1 --2- 172.21.15.43:0/191282333 >> v2:172.21.15.43:6800/588375333 conn(0x7f6e3c06ef80 0x7f6e3c071430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007fd0 tx=0x7f6e4800b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:31:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:34.949+0000 7f6e3b7fe640 1 -- 172.21.15.43:0/191282333 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6e4407b0f0 con 0x7f6e540fb440 2024-09-11T08:31:35.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.151+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6e5410bd00 con 0x7f6e540fb440 2024-09-11T08:31:35.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.151+0000 7f6e3b7fe640 1 -- 172.21.15.43:0/191282333 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6e4407aa90 con 0x7f6e540fb440 2024-09-11T08:31:35.152 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T08:31:35.152 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T08:31:35.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 >> v2:172.21.15.43:6800/588375333 conn(0x7f6e3c06ef80 msgr2=0x7f6e3c071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 --2- 172.21.15.43:0/191282333 >> v2:172.21.15.43:6800/588375333 conn(0x7f6e3c06ef80 0x7f6e3c071430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007fd0 tx=0x7f6e4800b040 comp rx=0 tx=0).stop 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e540fb440 msgr2=0x7f6e54114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e540fb440 0x7f6e54114f10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e4402a4a0 tx=0x7f6e44009da0 comp rx=0 tx=0).stop 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 shutdown_connections 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 --2- 172.21.15.43:0/191282333 >> v2:172.21.15.43:6800/588375333 conn(0x7f6e3c06ef80 0x7f6e3c071430 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e541031b0 0x7f6e5410f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e54102810 0x7f6e54115450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 --2- 172.21.15.43:0/191282333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e540fb440 0x7f6e54114f10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 >> 172.21.15.43:0/191282333 conn(0x7f6e54074c80 msgr2=0x7f6e540fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 shutdown_connections 2024-09-11T08:31:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T08:31:35.154+0000 7f6e594a1640 1 -- 172.21.15.43:0/191282333 wait complete. 2024-09-11T08:31:35.370 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:35 smithi172 ceph-mon[32987]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:35.370 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/191282333' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:31:35.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:35 smithi043 ceph-mon[40592]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:35.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/191282333' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:31:35.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:35 smithi043 ceph-mon[29355]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:35.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/191282333' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:31:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:31:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:31:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:31:35.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:31:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:31:35.829 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T08:31:35.830 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T08:31:35.830 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T08:31:35.844 INFO:tasks.workunit:Pulling workunits from ref d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:31:35.845 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T08:31:35.845 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:31:35.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:31:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-11T08:31:35.862 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:31:35.919 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-11T08:31:35.919 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-11T08:31:35.975 INFO:tasks.workunit:timeout=3h 2024-09-11T08:31:35.975 INFO:tasks.workunit:cleanup=True 2024-09-11T08:31:35.976 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:31:36.033 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T08:31:36.478 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:36 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:36.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:36 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:36.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:36 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:37.448 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:37 smithi172 ceph-mon[32987]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:37 smithi043 ceph-mon[40592]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:37 smithi043 ceph-mon[29355]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:37.645 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:37 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:37.248Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002172419s 2024-09-11T08:31:38.231 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:38 smithi172 podman[66394]: 2024-09-11 08:31:37.962233641 +0000 UTC m=+0.015603651 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-11T08:31:38.981 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:38 smithi172 podman[66394]: 2024-09-11 08:31:38.509454257 +0000 UTC m=+0.562824302 container create fe61939015f31635142f6cf4d93c76afb402d8e3bb8a318c00e6b6cc1597b220 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a, architecture=x86_64, vendor=Red Hat, Inc., release=239.1651231664, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, maintainer=Paul Cuzner , version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, vcs-type=git, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-09-11T08:31:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:39 smithi172 ceph-mon[32987]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:39 smithi043 ceph-mon[40592]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:39 smithi043 ceph-mon[29355]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:31:40.609 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 podman[66394]: 2024-09-11 08:31:40.357689399 +0000 UTC m=+2.411059399 container init fe61939015f31635142f6cf4d93c76afb402d8e3bb8a318c00e6b6cc1597b220 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , description=Ceph Grafana Container, io.openshift.expose-services=, build-date=2022-04-29T11:48:11.055897, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, io.buildah.version=1.32.0, vendor=Red Hat, Inc., architecture=x86_64, distribution-scope=public) 2024-09-11T08:31:40.609 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 podman[66394]: 2024-09-11 08:31:40.363612368 +0000 UTC m=+2.416982367 container start fe61939015f31635142f6cf4d93c76afb402d8e3bb8a318c00e6b6cc1597b220 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , release=239.1651231664, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, distribution-scope=public, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, name=ubi8) 2024-09-11T08:31:40.609 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.525920608Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T08:31:40.609 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526098899Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T08:31:40.609 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526118514Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T08:31:40.609 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526126552Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T08:31:40.609 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526132809Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T08:31:40.610 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526138672Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T08:31:40.610 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526144293Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T08:31:40.610 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526150157Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T08:31:40.610 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526156409Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T08:31:40.610 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526162169Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T08:31:40.610 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526168588Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526173996Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526180356Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=settings t=2024-09-11T08:31:40.526186167Z level=info msg="App mode production" 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore t=2024-09-11T08:31:40.526239654Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore t=2024-09-11T08:31:40.52625848Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.527103169Z level=info msg="Starting DB migrations" 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.527181998Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.527719692Z level=info msg="Executing migration" id="create user table" 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.528186494Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.528658611Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.529110072Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.529747034Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T08:31:40.611 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.530161372Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.530839173Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.531259413Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.531634821Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.532080924Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.532439318Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.53277556Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.53320359Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.533499223Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.533934593Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.534300435Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.534707833Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.535141736Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T08:31:40.612 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.535542574Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.536670567Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.537120981Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.537554799Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.537990217Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.538405822Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.538799209Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.539108154Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.539473088Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.539836496Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.540225599Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.540591332Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.541249571Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T08:31:40.613 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.541662534Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.542081407Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.542483986Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.542892368Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.543318504Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.543641337Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.544019268Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.544361541Z level=info msg="Executing migration" id="create star table" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.544710838Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.545142799Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.545529646Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.545975244Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.546346937Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T08:31:40.614 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.546751107Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.547207558Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.547593369Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.548436868Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.549267087Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.550045515Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.550882772Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.551900985Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.552562744Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.553445897Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.554291726Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.555146759Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.556004867Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T08:31:40.615 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.55695969Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T08:31:40.616 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.557724848Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T08:31:40.616 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.558610687Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T08:31:40.616 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.559586807Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T08:31:40.616 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.56223585Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T08:31:40.616 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.565012422Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T08:31:40.616 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.567839501Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T08:31:40.616 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.570783141Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.573486679Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.574208825Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.576987558Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.579794023Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.582720125Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.592167564Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.592492824Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.595369578Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.598236608Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.600814759Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T08:31:40.617 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.60384287Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T08:31:40.618 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.606531382Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T08:31:40.618 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.609307695Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T08:31:40.859 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.611945791Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T08:31:40.859 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.614738086Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T08:31:40.859 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.617530807Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T08:31:40.859 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.620135915Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.622820351Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.625512179Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.62805379Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.630769028Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.633455069Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.636069782Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.638751372Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.641476461Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.644161046Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.646944256Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T08:31:40.860 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.649527531Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.652271431Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.652933075Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.653793375Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.654597215Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.655435295Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.656378337Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.657483002Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.658412868Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.659131953Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.66009179Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.660827459Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.661786497Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T08:31:40.861 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.662474847Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.663377798Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.66386344Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.664383598Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.665206808Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.665525278Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.666084842Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.666938235Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.667794958Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.668707799Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T08:31:40.862 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.669658041Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.670514208Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.671257957Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.672233879Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.67294586Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.673953703Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.674656459Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.675681536Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.676105736Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.67697402Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.677885497Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T08:31:40.864 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.678835196Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.679156996Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.680005155Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.680290624Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.682883926Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.683633092Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.685607198Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.686171774Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.689077168Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.691557653Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.694450032Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.694888061Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T08:31:40.865 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.697699012Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.700636127Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.703316732Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.70360969Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.706564445Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.709507792Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.712209936Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.712565323Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.715301121Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.718181866Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.720839769Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.721417824Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T08:31:40.866 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.724264059Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.726846586Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.729483908Z level=info msg="Executing migration" id="create session table" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.729962078Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.732775883Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.735349575Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.737886817Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.738480395Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.741127325Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.741479757Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.74390053Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.744436805Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T08:31:40.867 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.746600987Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.74694688Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.747758722Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.748523431Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.749361561Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.749982453Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.750897817Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.751634463Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.752603358Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.753110331Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.753927538Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.754734744Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.755547242Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T08:31:40.868 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.755969952Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.756757255Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.757583938Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.759059573Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.759459676Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.760318211Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.76115443Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.762017026Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.762431529Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.763315255Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.764283498Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.765117979Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.765770202Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T08:31:40.869 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.766601351Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.767446964Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.768337815Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.768783339Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.769635669Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.770621503Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.771384796Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.771832114Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.772488135Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.781538469Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.78240981Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.785299925Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.785896339Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T08:31:40.870 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.788710276Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.791633227Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.794356334Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.794815028Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.797442775Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.8002948Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.803023031Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.803497357Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.806218898Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.809100731Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.811888697Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.812234608Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.814929368Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.817833118Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T08:31:40.871 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.820359547Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.820779273Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.823533082Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.826197938Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.828941496Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.829483709Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.832278719Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.834864767Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.837341829Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.837682127Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.840487748Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.843099614Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.845859516Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T08:31:40.872 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.846285926Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.848967855Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.849660734Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.851902198Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.854482259Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.855018641Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.857724659Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.858605862Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.859263215Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.859702143Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.860534015Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.861346561Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.862136427Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.862549042Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T08:31:40.873 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.862976024Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T08:31:41.111 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.863795499Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T08:31:41.111 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.864602348Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T08:31:41.111 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.864941974Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T08:31:41.111 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.865877859Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T08:31:41.111 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.866561233Z level=info msg="Executing migration" id="create team table" 2024-09-11T08:31:41.111 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.867517515Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.868084656Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.86890337Z level=info msg="Executing migration" id="create team member table" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.869819919Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.870544671Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.871042057Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.871915058Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.872794167Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.873639458Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.874402841Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.875217389Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T08:31:41.112 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.876001772Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.876510849Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.876986633Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.877851263Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.878607269Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.879080941Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.879527151Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.8799703Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.880315583Z level=info msg="Executing migration" id="create tag table" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.880732817Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.881210948Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.881774636Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.882236413Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T08:31:41.113 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.882671865Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T08:31:41.114 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.884931921Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T08:31:41.114 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.885362781Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T08:31:41.114 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.88583522Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T08:31:41.114 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.886184333Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T08:31:41.114 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.886553433Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T08:31:41.114 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.886977541Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T08:31:41.114 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.887420796Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T08:31:41.114 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.8877184Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T08:31:41.116 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.88848366Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T08:31:41.116 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.889254982Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T08:31:41.116 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.890055604Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.890837868Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.891293659Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.892087159Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.892502411Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.893013864Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.893474668Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.894307984Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.894825364Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.895281918Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.896087367Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T08:31:41.117 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.896530132Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.896995205Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.897449755Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.898227724Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.898576667Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.898923211Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.899416448Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.900228289Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.900729317Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.901107458Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T08:31:41.118 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.901529612Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.902004911Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.902484196Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.902954621Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.903773964Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.904238301Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.904549321Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.905008971Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.905525629Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.905986003Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.906335704Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.906761619Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T08:31:41.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.907245824Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.907662078Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.908172628Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.909030184Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.909491068Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.909987909Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.913515221Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.917061504Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.917548419Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.918344465Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.91920423Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.919637159Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.920487806Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T08:31:41.120 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.921371565Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.922226539Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.925041465Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.927796825Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.930813674Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.933510139Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.936248353Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.936712818Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.937566642Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.938445047Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.941248697Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.94394832Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.946819045Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.949675848Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T08:31:41.121 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.952534485Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.955193757Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.958022152Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.960821495Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.96367359Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.966394421Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.96916758Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.971957847Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.974867367Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.97782628Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.980443288Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T08:31:41.122 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.983338198Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.986036069Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.988786959Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.991543836Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.994219134Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.997125983Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:40 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:40.999752392Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.002574259Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.005192084Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.007900904Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.010552024Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.013311884Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.015984916Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.020010301Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T08:31:41.123 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.021247626Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.021360674Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.024103597Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.026856708Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.029481085Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.029657317Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.029827087Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.032356516Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.034989473Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.03578437Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.03668984Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.037437126Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.038268588Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.039054543Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T08:31:41.124 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.039929356Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.040774283Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.041567345Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.045931367Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.046883406Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.047703713Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.052047984Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.05641669Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.057478251Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.058104881Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.058860604Z level=info msg="Executing migration" id="create permission table" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.059790809Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T08:31:41.125 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.060685599Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.061559226Z level=info msg="Executing migration" id="create role table" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.062262053Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.063223487Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.064154842Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.065023861Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.06587574Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.066816141Z level=info msg="Executing migration" id="create team role table" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.067609013Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.06855453Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.069313029Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.070149664Z level=info msg="Executing migration" id="create user role table" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.071027671Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T08:31:41.126 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.071994419Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.072794043Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.073660843Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.074520801Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.075454725Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.078159752Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.080955254Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.083783113Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.086562692Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.089272416Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.092199777Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.094687313Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.097413424Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.100110738Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T08:31:41.127 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.103045411Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T08:31:41.128 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.105847387Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T08:31:41.128 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.108601044Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T08:31:41.128 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.111284845Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T08:31:41.128 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.114155551Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T08:31:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-mon[32987]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:41.481 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.11703666Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.119624957Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.122562984Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.125365296Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.1281855Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.130737563Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.133539906Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.136258952Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.138896927Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.141737222Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.144299748Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.147058948Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T08:31:41.482 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.149818602Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:31:41.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.150075777Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:31:41.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.152487411Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:31:41.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.152598111Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:31:41.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.155243218Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T08:31:41.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.157777173Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T08:31:41.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.160503864Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.162965714Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.165764391Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.16858087Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.171256929Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 bash[66394]: fe61939015f31635142f6cf4d93c76afb402d8e3bb8a318c00e6b6cc1597b220 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.172000476Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.172705036Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.173542715Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 systemd[1]: Started Ceph grafana.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.174375302Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.175153883Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.179998082Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T08:31:41.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.181101892Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.182123634Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.182985431Z level=info msg="Executing migration" id="add share column" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.184231262Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.185084739Z level=info msg="Executing migration" id="create file table" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.18585215Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.186739805Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.187627361Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.188440556Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.189226202Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.190088214Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.190902453Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.193214404Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T08:31:41.485 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.194009647Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.19520609Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.195942103Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.19680271Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.197582364Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.198361628Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.199247233Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.200340905Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.20137705Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.202142653Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.213077803Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.213863935Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.214715522Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T08:31:41.486 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.21866665Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.219044187Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.219873409Z level=info msg="Executing migration" id="create folder table" 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.220644449Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.221140207Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.221980538Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.222830591Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=migrator t=2024-09-11T08:31:41.223651339Z level=info msg="migrations completed" performed=485 skipped=0 duration=696.494386ms 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore t=2024-09-11T08:31:41.224108998Z level=info msg="Created default organization" 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=plugin.loader t=2024-09-11T08:31:41.285916459Z level=info msg="Plugin registered" pluginID=input 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=plugin.loader t=2024-09-11T08:31:41.313691335Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=plugin.loader t=2024-09-11T08:31:41.313728212Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T08:31:41.487 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=secrets t=2024-09-11T08:31:41.31377014Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T08:31:41.488 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=query_data t=2024-09-11T08:31:41.315470097Z level=info msg="Query Service initialization" 2024-09-11T08:31:41.488 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=live.push_http t=2024-09-11T08:31:41.317915439Z level=info msg="Live Push Gateway initialization" 2024-09-11T08:31:41.582 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:41 smithi043 ceph-mon[40592]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:41.582 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:41 smithi043 ceph-mon[29355]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:42.243 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:42 smithi043 bash[76750]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:31:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: Deploying daemon node-exporter.a on smithi043 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=infra.usagestats.collector t=2024-09-11T08:31:42.061401288Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=provisioning.datasources t=2024-09-11T08:31:42.061850478Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=provisioning.datasources t=2024-09-11T08:31:42.094271287Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=provisioning.datasources t=2024-09-11T08:31:42.126114362Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=provisioning.alerting t=2024-09-11T08:31:42.147429118Z level=info msg="starting to provision alerting" 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=provisioning.alerting t=2024-09-11T08:31:42.147448415Z level=info msg="finished to provision alerting" 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=grafanaStorageLogger t=2024-09-11T08:31:42.14755601Z level=info msg="storage starting" 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=ngalert.state.manager t=2024-09-11T08:31:42.147707678Z level=info msg="Warming state cache for startup" 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=http.server t=2024-09-11T08:31:42.152769881Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T08:31:42.483 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore.transactions t=2024-09-11T08:31:42.158891636Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:31:42.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=ngalert.state.manager t=2024-09-11T08:31:42.159049779Z level=info msg="State cache has been initialized" states=0 duration=11.340952ms 2024-09-11T08:31:42.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=ticker t=2024-09-11T08:31:42.159098861Z level=info msg=starting first_tick=2024-09-11T08:31:50Z 2024-09-11T08:31:42.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=ngalert.multiorg.alertmanager t=2024-09-11T08:31:42.159114821Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T08:31:42.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore.transactions t=2024-09-11T08:31:42.196985174Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:31:42.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore.transactions t=2024-09-11T08:31:42.227931518Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:31:42.484 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:31:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore.transactions t=2024-09-11T08:31:42.239189599Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-11T08:31:42.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: Deploying daemon node-exporter.a on smithi043 2024-09-11T08:31:42.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2024-09-11T08:31:42.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2024-09-11T08:31:42.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:42.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.543 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: Deploying daemon node-exporter.a on smithi043 2024-09-11T08:31:42.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2024-09-11T08:31:42.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-11T08:31:42.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]: dispatch 2024-09-11T08:31:42.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:42.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:42.894 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:42 smithi043 bash[76750]: Getting image source signatures 2024-09-11T08:31:42.894 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:42 smithi043 bash[76750]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:31:42.894 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:42 smithi043 bash[76750]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:31:42.894 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:42 smithi043 bash[76750]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:31:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[29355]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-11T08:31:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-11T08:31:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]': finished 2024-09-11T08:31:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[29355]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:31:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[40592]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-11T08:31:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-11T08:31:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]': finished 2024-09-11T08:31:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:43 smithi043 ceph-mon[40592]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:31:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:43 smithi172 ceph-mon[32987]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:31:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-11T08:31:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-11T08:31:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 5]}]': finished 2024-09-11T08:31:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:43 smithi172 ceph-mon[32987]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:31:44.272 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:43 smithi043 bash[76750]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:31:44.272 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 bash[76750]: Writing manifest to image destination 2024-09-11T08:31:44.625 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-mon[40592]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:31:44.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-mon[29355]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:31:44.625 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 podman[76750]: 2024-09-11 08:31:44.308131124 +0000 UTC m=+2.234842935 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:31:44.625 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 podman[76750]: 2024-09-11 08:31:44.398398375 +0000 UTC m=+2.325110178 container create 97777e91719481730cdcea91cdc4aca1fb6bf2ed7ed8d04d24fc7950e1c009cc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:44.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:44 smithi172 ceph-mon[32987]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:31:44.894 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 podman[76750]: 2024-09-11 08:31:44.62562728 +0000 UTC m=+2.552339080 container init 97777e91719481730cdcea91cdc4aca1fb6bf2ed7ed8d04d24fc7950e1c009cc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 podman[76750]: 2024-09-11 08:31:44.630885915 +0000 UTC m=+2.557597715 container start 97777e91719481730cdcea91cdc4aca1fb6bf2ed7ed8d04d24fc7950e1c009cc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.635Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.635Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.635Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.635Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.636Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.636Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:31:44.895 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:31:44.896 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:31:44.896 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:31:44.896 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:31:44.896 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:31:44.896 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:31:44.896 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:31:44.896 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:31:44.897 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:31:44.898 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.637Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.638Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a[76825]: ts=2024-09-11T08:31:44.638Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 bash[76750]: 97777e91719481730cdcea91cdc4aca1fb6bf2ed7ed8d04d24fc7950e1c009cc 2024-09-11T08:31:44.899 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:31:44 smithi043 systemd[1]: Started Ceph node-exporter.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:31:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:45 smithi172 ceph-mon[32987]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:31:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:45 smithi172 ceph-mon[32987]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-11T08:31:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:45 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:45 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:45 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[40592]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:31:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[40592]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-11T08:31:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[29355]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:31:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[29355]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-11T08:31:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:45 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:45.704 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:31:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:31:45.704 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:31:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:31:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:31:45.704 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:45 smithi172 systemd[1]: Starting Ceph node-exporter.b for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:31:45.981 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:45 smithi172 bash[66681]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:31:46.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:46 smithi172 ceph-mon[32987]: Deploying daemon node-exporter.b on smithi172 2024-09-11T08:31:46.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:46 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:46.481 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:46 smithi172 bash[66681]: Getting image source signatures 2024-09-11T08:31:46.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:46 smithi172 bash[66681]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:31:46.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:46 smithi172 bash[66681]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:31:46.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:46 smithi172 bash[66681]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:31:46.556 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:46 smithi043 ceph-mon[40592]: Deploying daemon node-exporter.b on smithi172 2024-09-11T08:31:46.556 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:46 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:46.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:46 smithi043 ceph-mon[29355]: Deploying daemon node-exporter.b on smithi172 2024-09-11T08:31:46.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:46 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:47.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:47 smithi172 ceph-mon[32987]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:31:47.540 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:47 smithi172 bash[66681]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:31:47.540 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:47 smithi172 bash[66681]: Writing manifest to image destination 2024-09-11T08:31:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:47 smithi043 ceph-mon[40592]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:31:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:47 smithi043 ceph-mon[29355]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:31:48.086 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:47 smithi172 podman[66681]: 2024-09-11 08:31:47.787112162 +0000 UTC m=+2.176616682 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:31:48.087 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:47 smithi172 podman[66681]: 2024-09-11 08:31:47.852213554 +0000 UTC m=+2.241718073 container create 7e793bfb3ce6b09033cdca599ea82604f46fb778ffa1c246367b28d9fee05555 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:31:48.481 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 podman[66681]: 2024-09-11 08:31:48.087354674 +0000 UTC m=+2.476859194 container init 7e793bfb3ce6b09033cdca599ea82604f46fb778ffa1c246367b28d9fee05555 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 podman[66681]: 2024-09-11 08:31:48.092274097 +0000 UTC m=+2.481778611 container start 7e793bfb3ce6b09033cdca599ea82604f46fb778ffa1c246367b28d9fee05555 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.096Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.096Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.096Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.096Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:31:48.482 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:31:48.483 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:31:48.483 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:31:48.483 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:31:48.483 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:31:48.483 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:31:48.483 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:31:48.483 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:31:48.484 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:31:48.485 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b[66755]: ts=2024-09-11T08:31:48.097Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 bash[66681]: 7e793bfb3ce6b09033cdca599ea82604f46fb778ffa1c246367b28d9fee05555 2024-09-11T08:31:48.486 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:31:48 smithi172 systemd[1]: Started Ceph node-exporter.b for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:31:49.263 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-09-11T08:31:49.263 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.263 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.263 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: Cluster is now healthy 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-11T08:31:49.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: Cluster is now healthy 2024-09-11T08:31:49.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:49 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:31:49.517 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-09-11T08:31:49.517 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.517 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.518 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.518 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.518 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:49.518 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-11T08:31:49.518 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: Cluster is now healthy 2024-09-11T08:31:49.518 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:49 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:31:51.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:51 smithi172 ceph-mon[32987]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-09-11T08:31:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:51 smithi043 ceph-mon[40592]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-09-11T08:31:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:51 smithi043 ceph-mon[29355]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-09-11T08:31:52.624 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:52 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:52.624 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:52 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:52.624 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:52 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:52 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:52 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:52 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:52.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:52 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:52 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:52 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.636 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[40592]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 930 B/s rd, 0 op/s; 66 B/s, 1 objects/s recovering 2024-09-11T08:31:53.636 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.637 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.637 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:31:53.637 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:31:53.637 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[29355]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 930 B/s rd, 0 op/s; 66 B/s, 1 objects/s recovering 2024-09-11T08:31:53.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:31:53.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:31:53.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:53 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:53 smithi172 ceph-mon[32987]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 930 B/s rd, 0 op/s; 66 B/s, 1 objects/s recovering 2024-09-11T08:31:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:53.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:53 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:31:53.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:53 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:31:53.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:53 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:54.339 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:54 smithi043 ceph-mon[29355]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:31:54.339 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:54 smithi043 ceph-mon[29355]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-11T08:31:54.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:54 smithi043 ceph-mon[40592]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:31:54.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:54 smithi043 ceph-mon[40592]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-11T08:31:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:54 smithi172 ceph-mon[32987]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:31:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:54 smithi172 ceph-mon[32987]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-11T08:31:55.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:55 smithi043 ceph-mon[40592]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 281 B/s wr, 5 op/s; 60 B/s, 1 objects/s recovering 2024-09-11T08:31:55.413 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:55 smithi043 ceph-mon[29355]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 281 B/s wr, 5 op/s; 60 B/s, 1 objects/s recovering 2024-09-11T08:31:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:55 smithi172 ceph-mon[32987]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 281 B/s wr, 5 op/s; 60 B/s, 1 objects/s recovering 2024-09-11T08:31:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:31:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:31:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:31:55.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:31:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:31:56.344 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:56 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:56 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:56.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:56 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:31:57.144 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:57 smithi043 systemd[1]: Stopping Ceph alertmanager.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:31:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:57 smithi043 ceph-mon[40592]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 255 B/s wr, 4 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T08:31:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:57 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:57.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:57 smithi043 ceph-mon[29355]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 255 B/s wr, 4 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T08:31:57.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:57 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:57.645 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:57 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[75759]: ts=2024-09-11T08:31:57.319Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:31:57.645 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:57 smithi043 podman[77732]: 2024-09-11 08:31:57.360064727 +0000 UTC m=+0.183117477 container died f826b14b35612e6b5da925665eb911fb133f467ca02385eb41e2e2f7ae557bee (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:57 smithi172 ceph-mon[32987]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 255 B/s wr, 4 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T08:31:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:31:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:57 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:31:58.022 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:57 smithi043 podman[77732]: 2024-09-11 08:31:57.76321018 +0000 UTC m=+0.586262923 container cleanup f826b14b35612e6b5da925665eb911fb133f467ca02385eb41e2e2f7ae557bee (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:58.023 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:57 smithi043 bash[77732]: ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:31:58.353 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:58 smithi043 podman[77750]: 2024-09-11 08:31:58.023280279 +0000 UTC m=+0.659588851 container remove f826b14b35612e6b5da925665eb911fb133f467ca02385eb41e2e2f7ae557bee (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:58.353 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:58 smithi043 podman[77750]: 2024-09-11 08:31:58.124318108 +0000 UTC m=+0.760626701 volume remove 7c42a2b4b33bb7df0170ec20e603a0c0df918f41d6c90e4dd433c7a4c1cad2d4 2024-09-11T08:31:58.616 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:58 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:31:58.616 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:58 smithi043 systemd[1]: Stopped Ceph alertmanager.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:31:58.616 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:58 smithi043 systemd[1]: Starting Ceph alertmanager.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:31:58.616 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:58 smithi043 podman[77846]: 2024-09-11 08:31:58.490256753 +0000 UTC m=+0.018505864 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T08:31:58.616 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:58 smithi043 podman[77846]: 2024-09-11 08:31:58.61703682 +0000 UTC m=+0.145285933 volume create 38df3fcc48ed56aa9a3f700af7bcf8b081e94ba697fadee421730d31019b743c 2024-09-11T08:31:58.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:31:58 smithi043 podman[77846]: 2024-09-11 08:31:58.719635635 +0000 UTC m=+0.247884746 container create 9c3e45c5a953ff5c13f624a10b7a80c1a41633ea24c6ba0aff7c7ba289b37900 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:31:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:31:59 smithi043 ceph-mon[40592]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T08:31:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:31:59 smithi043 ceph-mon[29355]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T08:31:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:31:59 smithi172 ceph-mon[32987]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T08:32:00.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 podman[77846]: 2024-09-11 08:32:00.540925774 +0000 UTC m=+2.069174885 container init 9c3e45c5a953ff5c13f624a10b7a80c1a41633ea24c6ba0aff7c7ba289b37900 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:32:00.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 podman[77846]: 2024-09-11 08:32:00.546515858 +0000 UTC m=+2.074764975 container start 9c3e45c5a953ff5c13f624a10b7a80c1a41633ea24c6ba0aff7c7ba289b37900 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:32:00.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:00.560Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:32:00.895 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:00.560Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:32:00.895 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:00.562Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-09-11T08:32:00.895 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:00.563Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:32:00.895 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:00.591Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:32:00.895 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:00.592Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:32:00.895 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:00.595Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:32:00.895 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:00.595Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:32:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:01 smithi043 ceph-mon[40592]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:01.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:01 smithi043 bash[77846]: 9c3e45c5a953ff5c13f624a10b7a80c1a41633ea24c6ba0aff7c7ba289b37900 2024-09-11T08:32:01.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:01 smithi043 systemd[1]: Started Ceph alertmanager.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:32:01.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:01 smithi043 ceph-mon[29355]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:01 smithi172 ceph-mon[32987]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:02 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:02 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:02 smithi172 ceph-mon[32987]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:32:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:02 smithi172 ceph-mon[32987]: Reconfiguring daemon prometheus.a on smithi172 2024-09-11T08:32:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-mon[40592]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:32:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-mon[40592]: Reconfiguring daemon prometheus.a on smithi172 2024-09-11T08:32:02.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:02.564Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000766923s 2024-09-11T08:32:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-mon[29355]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:32:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:02 smithi043 ceph-mon[29355]: Reconfiguring daemon prometheus.a on smithi172 2024-09-11T08:32:03.576 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:03 smithi172 ceph-mon[32987]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:03 smithi043 ceph-mon[40592]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:03.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:03 smithi043 ceph-mon[29355]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:04.481 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 systemd[1]: Stopping Ceph prometheus.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:32:04.909 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.644Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:32:04.909 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.644Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:32:04.909 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.644Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.644Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.644Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.644Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.644Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.644Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.645Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.645Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.645Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[64448]: ts=2024-09-11T08:32:04.645Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:32:04.910 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:04 smithi172 podman[67791]: 2024-09-11 08:32:04.655136486 +0000 UTC m=+0.106401946 container died f29653c1e60b9efeba0ef612977464615924215eaa826ae210c4b8968181b69e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:32:05.231 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 podman[67791]: 2024-09-11 08:32:05.017253571 +0000 UTC m=+0.468519030 container cleanup f29653c1e60b9efeba0ef612977464615924215eaa826ae210c4b8968181b69e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:32:05.231 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 bash[67791]: ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:32:05.644 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-mon[32987]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:05.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:32:05.645 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 podman[67811]: 2024-09-11 08:32:05.276631148 +0000 UTC m=+0.622983989 container remove f29653c1e60b9efeba0ef612977464615924215eaa826ae210c4b8968181b69e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:32:05.645 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T08:32:05.645 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 systemd[1]: Stopped Ceph prometheus.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:32:05.645 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.129s CPU time. 2024-09-11T08:32:05.645 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 systemd[1]: Starting Ceph prometheus.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:32:05.695 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:05 smithi043 ceph-mon[29355]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:05.696 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:05 smithi043 ceph-mon[40592]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 podman[67906]: 2024-09-11 08:32:05.644262671 +0000 UTC m=+0.110855444 container create 575201f09b8551343ddfe545825581d2351fa6f53345fe9a15e52b892338ee37 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 podman[67906]: 2024-09-11 08:32:05.550140529 +0000 UTC m=+0.016733317 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 podman[67906]: 2024-09-11 08:32:05.854913161 +0000 UTC m=+0.321505938 container init 575201f09b8551343ddfe545825581d2351fa6f53345fe9a15e52b892338ee37 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 podman[67906]: 2024-09-11 08:32:05.859729744 +0000 UTC m=+0.326322520 container start 575201f09b8551343ddfe545825581d2351fa6f53345fe9a15e52b892338ee37 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.883Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.883Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.883Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi172 (none))" 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.883Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.883Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:32:05.905 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.884Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.885Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.886Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.886Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.888Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.888Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.03µs 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.888Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.888Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.888Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.888Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=29.169µs wal_replay_duration=484.33µs wbl_replay_duration=98ns total_replay_duration=542.097µs 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.889Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.889Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.889Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:32:05.906 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.904Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.861901ms db_storage=741ns remote_storage=1.187µs web_handler=360ns query_engine=508ns scrape=386.856µs scrape_sd=29.943µs notify=32.626µs notify_sd=7.548µs rules=14.02023ms tracing=5.957µs 2024-09-11T08:32:05.907 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.904Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:32:05.907 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:32:05.904Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:32:06.231 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 bash[67906]: 575201f09b8551343ddfe545825581d2351fa6f53345fe9a15e52b892338ee37 2024-09-11T08:32:06.231 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:32:05 smithi172 systemd[1]: Started Ceph prometheus.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:32:06.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:05 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:05] ENGINE Bus STOPPING 2024-09-11T08:32:06.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:32:06.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Bus STOPPED 2024-09-11T08:32:06.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Bus STARTING 2024-09-11T08:32:06.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Serving on http://:::9283 2024-09-11T08:32:06.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Bus STARTED 2024-09-11T08:32:06.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Bus STOPPING 2024-09-11T08:32:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi172.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi172.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:32:06.646 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Bus STOPPED 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Bus STARTING 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:32:06.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi172.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:32:06.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:32:06.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi172.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:32:06.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:32:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi172.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi172.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:06 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:32:07.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Serving on http://:::9283 2024-09-11T08:32:07.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Bus STARTED 2024-09-11T08:32:07.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:06 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:06] ENGINE Bus STOPPING 2024-09-11T08:32:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:32:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:32:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:32:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi172.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:32:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:32:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi172.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:32:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[40592]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:07.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:07] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:32:07.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:07] ENGINE Bus STOPPED 2024-09-11T08:32:07.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:07] ENGINE Bus STARTING 2024-09-11T08:32:07.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:07] ENGINE Serving on http://:::9283 2024-09-11T08:32:07.645 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:32:07] ENGINE Bus STARTED 2024-09-11T08:32:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:32:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:32:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:32:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi172.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:32:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:32:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi172.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:32:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:07 smithi043 ceph-mon[29355]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:07 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:32:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:07 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:32:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:07 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:32:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:07 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi172.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:32:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:07 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:32:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:07 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi172.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:32:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:07 smithi172 ceph-mon[32987]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:09.700 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:09 smithi172 ceph-mon[32987]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:09 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:09 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:09 smithi043 ceph-mon[40592]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:09 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:09 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:09 smithi043 ceph-mon[29355]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:09 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:09 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:10.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:32:10 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:32:10.567Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004630809s 2024-09-11T08:32:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:11 smithi172 ceph-mon[32987]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:11.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:11 smithi043 ceph-mon[40592]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:11 smithi043 ceph-mon[29355]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:12 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:12 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:32:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:32:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:12 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:32:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:12.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:32:13.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:13 smithi043 ceph-mon[29355]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:13 smithi043 ceph-mon[40592]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:13 smithi172 ceph-mon[32987]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:13] "GET /metrics HTTP/1.1" 200 219785 "" "Prometheus/2.43.0" 2024-09-11T08:32:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:15 smithi172 ceph-mon[32987]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:15.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:32:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:32:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:32:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:32:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:15 smithi043 ceph-mon[29355]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:15 smithi043 ceph-mon[40592]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:16 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:16 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:16 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:16 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:16 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:16 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:16.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:16 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:16 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:16 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[40592]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:32:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:32:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[29355]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:17.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:32:17.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:32:17.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:17 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:17 smithi172 ceph-mon[32987]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:17 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:17 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:17 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:32:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:17 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:32:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:17 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:19.783 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:19 smithi172 ceph-mon[32987]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:19 smithi043 ceph-mon[40592]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:19.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:19 smithi043 ceph-mon[29355]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:21 smithi043 ceph-mon[40592]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:21.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:21 smithi043 ceph-mon[29355]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:21.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:21 smithi172 ceph-mon[32987]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:23.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:23 smithi043 ceph-mon[40592]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:23.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:23] "GET /metrics HTTP/1.1" 200 219785 "" "Prometheus/2.43.0" 2024-09-11T08:32:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:23 smithi043 ceph-mon[29355]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:23 smithi172 ceph-mon[32987]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:25 smithi172 ceph-mon[32987]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:25.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:32:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:32:25.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:32:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:32:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:25 smithi043 ceph-mon[40592]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:25.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:25 smithi043 ceph-mon[29355]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:26 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:26.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:26 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:26.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:26 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:27 smithi043 ceph-mon[40592]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:27.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:27 smithi043 ceph-mon[29355]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:27.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:27 smithi172 ceph-mon[32987]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:29 smithi043 ceph-mon[40592]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:29 smithi043 ceph-mon[29355]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:29 smithi172 ceph-mon[32987]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:31 smithi043 ceph-mon[40592]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:31 smithi043 ceph-mon[29355]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:31 smithi172 ceph-mon[32987]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:33.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:33] "GET /metrics HTTP/1.1" 200 219752 "" "Prometheus/2.43.0" 2024-09-11T08:32:33.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:33 smithi043 ceph-mon[40592]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:33 smithi043 ceph-mon[29355]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:33 smithi172 ceph-mon[32987]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:35 smithi172 ceph-mon[32987]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:35.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:32:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:32:35.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:32:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:32:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:35 smithi043 ceph-mon[40592]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:35 smithi043 ceph-mon[29355]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:36 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:36.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:36 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:36 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:37.581 INFO:tasks.workunit.client.0.smithi043.stderr:Updating files: 90% (11472/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-11T08:32:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:37 smithi043 ceph-mon[40592]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:37 smithi043 ceph-mon[29355]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:37 smithi172 ceph-mon[32987]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:38.678 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'. 2024-09-11T08:32:38.678 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T08:32:38.678 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T08:32:38.678 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T08:32:38.678 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T08:32:38.679 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at d0e6828a201 Merge branch 'wip-63919-quincy' of https://github.com/guits/ceph into wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T08:32:38.685 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T08:32:38.710 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T08:32:38.710 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T08:32:38.710 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T08:32:39.465 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T08:32:39.552 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T08:32:39.594 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T08:32:39.596 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T08:32:39.596 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T08:32:39.636 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T08:32:39.640 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T08:32:39.640 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T08:32:39.696 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-11T08:32:39.698 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-11T08:32:39.698 DEBUG:teuthology.orchestra.run.smithi043:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d0e6828a2016d48cf25ad84064e50742bb1c39b9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-11T08:32:39.770 INFO:tasks.workunit.client.0.smithi043.stderr:+ parallel=1 2024-09-11T08:32:39.770 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' '' = --serial ']' 2024-09-11T08:32:39.770 INFO:tasks.workunit.client.0.smithi043.stderr:+ color= 2024-09-11T08:32:39.770 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -t 1 ']' 2024-09-11T08:32:39.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-11T08:32:39.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ declare -A pids 2024-09-11T08:32:39.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_aio 2024-09-11T08:32:39.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_aio' 2024-09-11T08:32:39.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_aio 2024-09-11T08:32:39.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.772 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_aio 2024-09-11T08:32:39.772 INFO:tasks.workunit.client.0.smithi043.stdout:test api_aio on pid 79021 2024-09-11T08:32:39.773 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79021 2024-09-11T08:32:39.773 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_aio on pid 79021' 2024-09-11T08:32:39.773 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79021 2024-09-11T08:32:39.773 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.773 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.773 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-11T08:32:39.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_aio_pp 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_aio_pp' 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_aio_pp 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_aio_pp 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79028 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_aio_pp on pid 79028' 2024-09-11T08:32:39.777 INFO:tasks.workunit.client.0.smithi043.stdout:test api_aio_pp on pid 79028 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79028 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_io 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_io' 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_io 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.786 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.786 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_io 2024-09-11T08:32:39.786 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79037 2024-09-11T08:32:39.786 INFO:tasks.workunit.client.0.smithi043.stdout:test api_io on pid 79037 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_io on pid 79037' 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79037 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_io_pp 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_io_pp' 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_io_pp 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_io_pp 2024-09-11T08:32:39.795 INFO:tasks.workunit.client.0.smithi043.stdout:test api_io_pp on pid 79049 2024-09-11T08:32:39.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79049 2024-09-11T08:32:39.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_io_pp on pid 79049' 2024-09-11T08:32:39.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79049 2024-09-11T08:32:39.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_asio 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_asio' 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_asio 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.808 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_asio 2024-09-11T08:32:39.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.808 INFO:tasks.workunit.client.0.smithi043.stdout:test api_asio on pid 79065 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79065 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_asio on pid 79065' 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79065 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_list 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_list' 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_list 2024-09-11T08:32:39.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.819 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79021/exe 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_list 2024-09-11T08:32:39.824 INFO:tasks.workunit.client.0.smithi043.stdout:test api_list on pid 79082 2024-09-11T08:32:39.829 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79082 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_list on pid 79082' 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79082 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.830 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_lock 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_lock' 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_lock 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.833 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_aio 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.835 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.840 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_aio: /' 2024-09-11T08:32:39.841 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.841 INFO:tasks.workunit.client.0.smithi043.stdout:test api_lock on pid 79101 2024-09-11T08:32:39.842 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_lock 2024-09-11T08:32:39.842 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79101 2024-09-11T08:32:39.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79028/exe 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_lock on pid 79101' 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79101 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_lock_pp 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_lock_pp' 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79037/exe 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_lock_pp 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_aio_pp 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.852 INFO:tasks.workunit.client.0.smithi043.stdout:test api_lock_pp on pid 79119 2024-09-11T08:32:39.854 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_lock_pp 2024-09-11T08:32:39.854 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.854 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79119 2024-09-11T08:32:39.854 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_lock_pp on pid 79119' 2024-09-11T08:32:39.854 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79119 2024-09-11T08:32:39.854 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_misc 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_misc' 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.855 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.857 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_io.log 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_io 2024-09-11T08:32:39.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_io: /' 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79049/exe 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_misc 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.863 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.863 INFO:tasks.workunit.client.0.smithi043.stdout:test api_misc on pid 79136 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_misc 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79136 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_misc on pid 79136' 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79136 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_misc_pp 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_misc_pp' 2024-09-11T08:32:39.867 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_io_pp 2024-09-11T08:32:39.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_misc_pp 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.873 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.879 INFO:tasks.workunit.client.0.smithi043.stdout:test api_misc_pp on pid 79157 2024-09-11T08:32:39.887 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_misc_pp 2024-09-11T08:32:39.887 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79157 2024-09-11T08:32:39.887 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_misc_pp on pid 79157' 2024-09-11T08:32:39.887 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79157 2024-09-11T08:32:39.887 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79065/exe 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_tier_pp 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_tier_pp' 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_asio 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_asio: /' 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79082/exe 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_tier_pp 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-11T08:32:39.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_list: /' 2024-09-11T08:32:39.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_list.log 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_list 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79101/exe 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.907 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.908 INFO:tasks.workunit.client.0.smithi043.stdout:test api_tier_pp on pid 79199 2024-09-11T08:32:39.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_tier_pp 2024-09-11T08:32:39.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79199 2024-09-11T08:32:39.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_tier_pp on pid 79199' 2024-09-11T08:32:39.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79199 2024-09-11T08:32:39.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_pool 2024-09-11T08:32:39.912 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_pool' 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.915 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_pool 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.918 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79119/exe 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_lock: /' 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-11T08:32:39.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_lock 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.925 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_pool 2024-09-11T08:32:39.925 INFO:tasks.workunit.client.0.smithi043.stdout:test api_pool on pid 79248 2024-09-11T08:32:39.928 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79248 2024-09-11T08:32:39.928 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_pool on pid 79248' 2024-09-11T08:32:39.928 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79248 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_snapshots 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_snapshots' 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_lock_pp 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_snapshots 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.939 INFO:tasks.workunit.client.0.smithi043.stdout:test api_snapshots on pid 79288 2024-09-11T08:32:39.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_snapshots 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79288 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_snapshots on pid 79288' 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79288 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.944 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_snapshots_pp 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_snapshots_pp' 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79136/exe 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.947 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79199/exe 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_misc 2024-09-11T08:32:39.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_misc: /' 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-11T08:32:39.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.958 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_tier_pp 2024-09-11T08:32:39.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_snapshots_pp 2024-09-11T08:32:39.958 INFO:tasks.workunit.client.0.smithi043.stdout:test api_snapshots_pp on pid 79328 2024-09-11T08:32:39.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.962 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.962 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_snapshots_pp 2024-09-11T08:32:39.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79328 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_snapshots_pp on pid 79328' 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79328 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79157/exe 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_stat 2024-09-11T08:32:39.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_stat' 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.966 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_misc_pp 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.969 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_stat 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:39.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_stat 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:39.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:39.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79378 2024-09-11T08:32:39.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_stat on pid 79378' 2024-09-11T08:32:39.973 INFO:tasks.workunit.client.0.smithi043.stdout:test api_stat on pid 79378 2024-09-11T08:32:39.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79378 2024-09-11T08:32:39.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_stat_pp 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_stat_pp' 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79248/exe 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:39.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:39.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.983 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.984 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_pool: /' 2024-09-11T08:32:39.984 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-11T08:32:39.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_stat_pp 2024-09-11T08:32:39.984 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_stat_pp 2024-09-11T08:32:39.984 INFO:tasks.workunit.client.0.smithi043.stdout:test api_stat_pp on pid 79414 2024-09-11T08:32:39.985 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79414 2024-09-11T08:32:39.985 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_stat_pp on pid 79414' 2024-09-11T08:32:39.985 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79414 2024-09-11T08:32:39.985 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_pool 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.986 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79288/exe 2024-09-11T08:32:39.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:39 smithi043 ceph-mon[40592]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:39.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:39 smithi043 ceph-mon[29355]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:39.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:39 smithi172 ceph-mon[32987]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:39.990 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-11T08:32:39.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_watch_notify 2024-09-11T08:32:39.990 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_watch_notify' 2024-09-11T08:32:39.990 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:39.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_snapshots 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:39.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_watch_notify 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:39.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:39.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:39.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:39.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:39.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:39.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:39.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:39.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:39.998 INFO:tasks.workunit.client.0.smithi043.stdout:test api_watch_notify on pid 79440 2024-09-11T08:32:40.000 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_watch_notify 2024-09-11T08:32:40.000 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79440 2024-09-11T08:32:40.000 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_watch_notify on pid 79440' 2024-09-11T08:32:40.000 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79440 2024-09-11T08:32:40.000 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.000 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.003 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_watch_notify_pp 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_watch_notify_pp' 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.004 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_watch_notify_pp 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.008 INFO:tasks.workunit.client.0.smithi043.stdout:test api_watch_notify_pp on pid 79462 2024-09-11T08:32:40.012 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_watch_notify_pp 2024-09-11T08:32:40.012 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79462 2024-09-11T08:32:40.012 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_watch_notify_pp on pid 79462' 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79462 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cmd 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cmd' 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cmd 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79328/exe 2024-09-11T08:32:40.025 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cmd on pid 79481 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cmd 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79481 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cmd on pid 79481' 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79481 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cmd_pp 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cmd_pp' 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_snapshots_pp 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.036 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.037 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.038 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79378/exe 2024-09-11T08:32:40.039 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.040 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_stat 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.044 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.047 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-11T08:32:40.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cmd_pp 2024-09-11T08:32:40.047 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_stat: /' 2024-09-11T08:32:40.047 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cmd_pp on pid 79552 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cmd_pp 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79552 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cmd_pp on pid 79552' 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79552 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_service 2024-09-11T08:32:40.050 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79414/exe 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_service' 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.051 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_service 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.053 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_service 2024-09-11T08:32:40.054 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79575 2024-09-11T08:32:40.055 INFO:tasks.workunit.client.0.smithi043.stdout:test api_service on pid 79575 2024-09-11T08:32:40.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_service on pid 79575' 2024-09-11T08:32:40.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79575 2024-09-11T08:32:40.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-11T08:32:40.058 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.058 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_service_pp 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_service_pp' 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_service_pp 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_stat_pp 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79440/exe 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.065 INFO:tasks.workunit.client.0.smithi043.stdout:test api_service_pp on pid 79596 2024-09-11T08:32:40.070 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_service_pp 2024-09-11T08:32:40.070 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79596 2024-09-11T08:32:40.070 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_service_pp on pid 79596' 2024-09-11T08:32:40.070 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79596 2024-09-11T08:32:40.070 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.070 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_c_write_operations 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_c_write_operations' 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.074 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.075 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_c_write_operations 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.078 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79462/exe 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.081 INFO:tasks.workunit.client.0.smithi043.stdout:test api_c_write_operations on pid 79616 2024-09-11T08:32:40.086 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_c_write_operations 2024-09-11T08:32:40.086 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79616 2024-09-11T08:32:40.086 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_c_write_operations on pid 79616' 2024-09-11T08:32:40.086 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79616 2024-09-11T08:32:40.086 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.086 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.086 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_c_read_operations 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_c_read_operations' 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.088 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.089 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_c_read_operations 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.090 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_watch_notify 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.091 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.092 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.093 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79481/exe 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.099 INFO:tasks.workunit.client.0.smithi043.stdout:test api_c_read_operations on pid 79644 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_c_read_operations 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79644 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_c_read_operations on pid 79644' 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79644 2024-09-11T08:32:40.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cls_remote_reads 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cls_remote_reads' 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cls_remote_reads 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.111 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.111 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cls_remote_reads on pid 79661 2024-09-11T08:32:40.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cls_remote_reads 2024-09-11T08:32:40.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79661 2024-09-11T08:32:40.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cls_remote_reads on pid 79661' 2024-09-11T08:32:40.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79661 2024-09-11T08:32:40.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.114 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s list_parallel 2024-09-11T08:32:40.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' list_parallel' 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.116 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cmd 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo list_parallel 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.122 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-11T08:32:40.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.122 INFO:tasks.workunit.client.0.smithi043.stdout:test list_parallel on pid 79694 2024-09-11T08:32:40.126 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=list_parallel 2024-09-11T08:32:40.126 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.126 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.126 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.126 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79694 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test list_parallel on pid 79694' 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79694 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.127 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.128 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cmd: /' 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.130 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s open_pools_parallel 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' open_pools_parallel' 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.133 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo open_pools_parallel 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.136 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.137 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.138 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.138 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.138 INFO:tasks.workunit.client.0.smithi043.stdout:test open_pools_parallel on pid 79728 2024-09-11T08:32:40.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=open_pools_parallel 2024-09-11T08:32:40.141 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79552/exe 2024-09-11T08:32:40.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79728 2024-09-11T08:32:40.141 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test open_pools_parallel on pid 79728' 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79728 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.145 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s delete_pools_parallel 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.146 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' delete_pools_parallel' 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.147 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.148 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.149 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79616/exe 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.152 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo delete_pools_parallel 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79575/exe 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.156 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.156 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.156 INFO:tasks.workunit.client.0.smithi043.stdout:test delete_pools_parallel on pid 79757 2024-09-11T08:32:40.169 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.169 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.169 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=delete_pools_parallel 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79757 2024-09-11T08:32:40.170 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test delete_pools_parallel on pid 79757' 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=79757 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:+ ret=0 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79199 2024-09-11T08:32:40.171 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79199 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_c_write_operations 2024-09-11T08:32:40.172 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cmd_pp 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79644/exe 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.173 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79596/exe 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.174 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.175 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_service 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_service: /' 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.176 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_service.log 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_service_pp 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.177 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.178 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_c_read_operations 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:32:40.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:32:40.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:32:40.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.189 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79661/exe 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:32:40.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.192 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.193 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.195 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:32:40.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79694/exe 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ list_parallel: /' 2024-09-11T08:32:40.201 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79728/exe 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:32:40.202 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_list_parallel 2024-09-11T08:32:40.203 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/79757/exe 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.204 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_open_pools_parallel 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:32:40.205 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-11T08:32:40.206 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-11T08:32:40.206 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-11T08:32:40.206 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-11T08:32:40.206 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_delete_pools_parallel 2024-09-11T08:32:41.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:41.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3545363988' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-11T08:32:41.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:41.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3545363988' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-11T08:32:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3545363988' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-11T08:32:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:43] "GET /metrics HTTP/1.1" 200 231381 "" "Prometheus/2.43.0" 2024-09-11T08:32:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/105021684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/140265294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:45 smithi172 ceph-mon[32987]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:32:45.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:32:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:32:45.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:32:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:32:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/105021684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/140265294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:45 smithi043 ceph-mon[29355]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:32:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/105021684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/140265294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:45 smithi043 ceph-mon[40592]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:32:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3670086344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1326783601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[40592]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:32:48.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3670086344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1326783601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[29355]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:32:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3670086344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1326783601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:48 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:48 smithi172 ceph-mon[32987]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:32:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3706316110' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1854050848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4189304441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/810327414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3433334863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/48450247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3706316110' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1854050848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4189304441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/810327414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3433334863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/48450247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3706316110' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1854050848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4189304441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/810327414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3433334863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/48450247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:50.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: pgmap v67: 292 pgs: 160 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: pgmap v68: 292 pgs: 138 creating+peering, 22 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: pgmap v70: 548 pgs: 138 creating+peering, 278 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: pgmap v71: 548 pgs: 345 creating+peering, 58 unknown, 145 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4151255400' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/67033807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3544464888' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1459046605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: pgmap v73: 772 pgs: 345 creating+peering, 282 unknown, 145 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:51.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1011071772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-79124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/749203886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79742-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1961800216' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: pgmap v67: 292 pgs: 160 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: pgmap v68: 292 pgs: 138 creating+peering, 22 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: pgmap v70: 548 pgs: 138 creating+peering, 278 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: pgmap v71: 548 pgs: 345 creating+peering, 58 unknown, 145 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4151255400' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/67033807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3544464888' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1459046605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: pgmap v73: 772 pgs: 345 creating+peering, 282 unknown, 145 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1011071772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-79124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/749203886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79742-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1961800216' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/672376336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/363371316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: Health check failed: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/672376336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/363371316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: Health check failed: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:51.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-11T08:32:51.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] Global test environment set-up. 2024-09-11T08:32:51.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-11T08:32:51.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (2 ms) 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (0 ms) 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (2 ms) 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (18 ms) 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-11T08:32:51.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (9 ms) 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (4 ms) 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (5 ms) 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (2 ms) 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (4 ms) 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (12 ms) 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-11T08:32:51.675 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (7 ms) 2024-09-11T08:32:51.676 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-11T08:32:51.676 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (5 ms) 2024-09-11T08:32:51.676 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] 12 tests from AsioRados (71 ms total) 2024-09-11T08:32:51.676 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: 2024-09-11T08:32:51.676 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] Global test environment tear-down 2024-09-11T08:32:51.676 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (11802 ms total) 2024-09-11T08:32:51.676 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-11T08:32:51.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: pgmap v67: 292 pgs: 160 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:32:51.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: pgmap v68: 292 pgs: 138 creating+peering, 22 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:51.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:51.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: pgmap v70: 548 pgs: 138 creating+peering, 278 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:32:51.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: pgmap v71: 548 pgs: 345 creating+peering, 58 unknown, 145 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:32:51.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4151255400' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/67033807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3544464888' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1459046605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: pgmap v73: 772 pgs: 345 creating+peering, 282 unknown, 145 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1011071772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-79124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/749203886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79742-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1961800216' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-79142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-79112-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-79089-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/672376336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/363371316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: Health check failed: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:51.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] Global test environment set-up. 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjects (327 ms) 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (32 ms) 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo2,foo3 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-09-11T08:32:52.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo2 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo3 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo4,foo5 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo4 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo5 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo6,foo7 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo7 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo6 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo4 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo5 2024-09-11T08:32:52.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo7 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo6 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo1 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo1 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo2 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo3 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (84 ms) 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 1 0 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 10 0 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 13 0 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 7 0 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 14 0 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 0 0 2024-09-11T08:32:52.008 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 15 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 11 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 5 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 8 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 6 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 3 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 4 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 12 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 9 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2 0 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-11T08:32:52.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (64 ms) 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: x cursor=MIN 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=1 cursor=12:02547ec2:::1:head 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=10 cursor=12:52ea6a34:::10:head 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=13 cursor=12:566253c9:::13:head 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=7 cursor=12:5c6b0b28:::7:head 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=14 cursor=12:62a1935d:::14:head 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=0 cursor=12:6cac518f:::0:head 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=15 cursor=12:863748b0:::15:head 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=11 cursor=12:89d3ae78:::11:head 2024-09-11T08:32:52.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=5 cursor=12:b29083e3:::5:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=8 cursor=12:bd63b0f1:::8:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=6 cursor=12:c4fdafeb:::6:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=3 cursor=12:cfc208b3:::3:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=4 cursor=12:d83876eb:::4:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=12 cursor=12:de5d7c5f:::12:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=9 cursor=12:e960b815:::9:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=2 cursor=12:f905c69b:::2:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=1 cursor=12:02547ec2:::1:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:02547ec2:::1:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:02547ec2:::1:head -> 1 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=10 cursor=12:52ea6a34:::10:head 2024-09-11T08:32:52.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:52ea6a34:::10:head 2024-09-11T08:32:52.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:52ea6a34:::10:head -> 10 2024-09-11T08:32:52.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=13 cursor=12:566253c9:::13:head 2024-09-11T08:32:52.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:566253c9:::13:head 2024-09-11T08:32:52.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:566253c9:::13:head -> 13 2024-09-11T08:32:52.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=7 cursor=12:5c6b0b28:::7:head 2024-09-11T08:32:52.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:5c6b0b28:::7:head 2024-09-11T08:32:52.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:5c6b0b28:::7:head -> 7 2024-09-11T08:32:52.075 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=14 cursor=12:62a1935d:::14:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:62a1935d:::14:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:62a1935d:::14:head -> 14 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=0 cursor=12:6cac518f:::0:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:6cac518f:::0:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:6cac518f:::0:head -> 0 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=15 cursor=12:863748b0:::15:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:863748b0:::15:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:863748b0:::15:head -> 15 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=11 cursor=12:89d3ae78:::11:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:89d3ae78:::11:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:89d3ae78:::11:head -> 11 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=5 cursor=12:b29083e3:::5:head 2024-09-11T08:32:52.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:b29083e3:::5:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:b29083e3:::5:head -> 5 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=8 cursor=12:bd63b0f1:::8:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:bd63b0f1:::8:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:bd63b0f1:::8:head -> 8 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=6 cursor=12:c4fdafeb:::6:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:c4fdafeb:::6:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:c4fdafeb:::6:head -> 6 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=3 cursor=12:cfc208b3:::3:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:cfc208b3:::3:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:cfc208b3:::3:head -> 3 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=4 cursor=12:d83876eb:::4:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:d83876eb:::4:head 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:d83876eb:::4:head -> 4 2024-09-11T08:32:52.077 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=12 cursor=12:de5d7c5f:::12:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:de5d7c5f:::12:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:de5d7c5f:::12:head -> 12 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=9 cursor=12:e960b815:::9:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:e960b815:::9:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:e960b815:::9:head -> 9 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=2 cursor=12:f905c69b:::2:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:f905c69b:::2:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:f905c69b:::2:head -> 2 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:02547ec2:::1:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:02547ec2:::1:head expected=12:02547ec2:::1:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:02547ec2:::1:head -> 1 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=1 expected=1 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:d83876eb:::4:head 2024-09-11T08:32:52.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:d83876eb:::4:head expected=12:d83876eb:::4:head 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:d83876eb:::4:head -> 4 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=4 expected=4 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:f905c69b:::2:head 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:f905c69b:::2:head expected=12:f905c69b:::2:head 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:f905c69b:::2:head -> 2 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=2 expected=2 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:e960b815:::9:head 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:e960b815:::9:head expected=12:e960b815:::9:head 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:e960b815:::9:head -> 9 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=9 expected=9 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:de5d7c5f:::12:head 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:de5d7c5f:::12:head expected=12:de5d7c5f:::12:head 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:de5d7c5f:::12:head -> 12 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=12 expected=12 2024-09-11T08:32:52.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:566253c9:::13:head 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:566253c9:::13:head expected=12:566253c9:::13:head 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:566253c9:::13:head -> 13 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=13 expected=13 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:cfc208b3:::3:head 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:cfc208b3:::3:head expected=12:cfc208b3:::3:head 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:cfc208b3:::3:head -> 3 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=3 expected=3 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:c4fdafeb:::6:head 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:c4fdafeb:::6:head expected=12:c4fdafeb:::6:head 2024-09-11T08:32:52.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:c4fdafeb:::6:head -> 6 2024-09-11T08:32:52.368 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=6 list_parallel: process_1_[79927]: starting. 2024-09-11T08:32:52.368 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[79927]: creating pool ceph_test_rados_list_parallel.smithi043-79902 2024-09-11T08:32:52.368 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[79927]: created object 0... 2024-09-11T08:32:52.368 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[79927]: created object 25... 2024-09-11T08:32:52.368 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[79927]: created object 49... 2024-09-11T08:32:52.368 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[79927]: finishing. 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[79927]: shutting down. 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[79928]: starting. 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[79928]: listing objects. 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[79928]: listed object 0... 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[79928]: listed object 25... 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[79928]: saw 50 objects 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[79928]: shutting down. 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[80585]: starting. 2024-09-11T08:32:52.369 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[80585]: creating pool ceph_test_rados_list_parallel.smithi043-79902 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[80585]: created object 0... 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[80585]: created object 25... 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[80585]: created object 49... 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[80585]: finishing. 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[80585]: shutting down. 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[80586]: starting. 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[80586]: listing objects. 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[80586]: listed object 0... 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[80586]: listed object 25... 2024-09-11T08:32:52.370 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[80586]: saw 43 objects 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[80586]: shutting down. 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[80587]: starting. 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[80587]: removed 25 objects... 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[80587]: removed half of the objects 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[80587]: removed 50 objects... 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[80587]: removed 50 objects 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[80587]: shutting down. 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.371 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[80658]: starting. 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[80658]: creating pool ceph_test_rados_list_parallel.smithi043-79902 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[80658]: created object 0... 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[80658]: created object 25... 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[80658]: created object 49... 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[80658]: finishing. 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[80658]: shutting down. 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[80659]: starting. 2024-09-11T08:32:52.372 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[80659]: listing objects. 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[80659]: listed object 0... 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[80659]: listed object 25... 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[80659]: listed object 50... 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[80659]: saw 56 objects 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[80659]: shutting down. 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[80660]: starting. 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[80660]: added 25 objects... 2024-09-11T08:32:52.373 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[80660]: added half of the objects 2024-09-11T08:32:52.374 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[80660]: added 50 objects... 2024-09-11T08:32:52.374 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[80660]: added 50 objects 2024-09-11T08:32:52.374 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[80660]: shutting down. 2024-09-11T08:32:52.374 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.374 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[80706]: starting. 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[80706]: creating pool ceph_test_rados_list_parallel.smithi043-79902 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[80706]: created object 0... 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[80706]: created object 25... 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[80706]: created object 49... 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[80706]: finishing. 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[80706]: shutting down. 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.760 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[80707]: starting. 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[80707]: listing objects. 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[80707]: listed object 0... 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[80707]: listed object 25... 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[80707]: listed object 50... 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[80707]: listed object 75... 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[80707]: saw 92 objects 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[80707]: shutting down. 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[80710]: starting. 2024-09-11T08:32:52.761 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[80710]: removed 25 objects... 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[80710]: removed half of the objects 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[80710]: removed 50 objects... 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[80710]: removed 50 objects 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[80710]: shutting down. 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[80708]: starting. 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[80708]: added 25 objects... 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[80708]: added half of the objects 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[80708]: added 50 objects... 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[80708]: added 50 objects 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[80708]: shutting down. 2024-09-11T08:32:52.762 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[80709]: starting. 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[80709]: added 25 objects... 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[80709]: added half of the objects 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[80709]: added 50 objects... 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[80709]: added 50 objects 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[80709]: shutting down. 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[80818]: starting. 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[80818]: creating pool ceph_test_rados_list_parallel.smithi043-79902 2024-09-11T08:32:52.763 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[80818]: created object 0... 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[80818]: created object 25... 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[80818]: created object 49... 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[80818]: finishing. 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[80818]: shutting down. 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[80820]: starting. 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[80820]: added 25 objects... 2024-09-11T08:32:52.764 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[80820]: added half of the objects 2024-09-11T08:32:52.989 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[80820]: added api_cmd_pp: Running main() from gmock_main.cc 2024-09-11T08:32:52.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-11T08:32:52.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (53 ms) 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (55 ms) 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (12848 ms) 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (12956 ms total) 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (12956 ms total) 2024-09-11T08:32:52.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-11T08:32:53.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1011071772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-79124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/749203886' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79742-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1011071772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-79124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/749203886' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79742-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:53.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:32:53.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:53.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:53.398 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:53.398 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:32:53.398 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:53.398 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1011071772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-79124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/749203886' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79742-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:32:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:53.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:53.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (13393 ms) 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (13393 ms total) 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (13393 ms total) 2024-09-11T08:32:53.499 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-09-11T08:32:53.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (85 ms) 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (703 ms) 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (12692 ms) 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-11T08:32:53.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.405977+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-79202-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406081+0000 mon.a [INF] from='client.? 172.21.15.43:0/1011071772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-79124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406270+0000 mon.a [INF] from='client.? 172.21.15.43:0/749203886' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79742-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406375+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-79173-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406461+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406573+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-79226-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406647+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406740+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-79529-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406817+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-79495-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406900+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-79424-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.406991+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-79572-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:53.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.407135+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79869-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:32:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:53] "GET /metrics HTTP/1.1" 200 231381 "" "Prometheus/2.43.0" 2024-09-11T08:32:54.502 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.407229+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWa api_c_read_operations: Running main() from gmock_main.cc 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (180 ms) 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (95 ms) 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (10 ms) 2024-09-11T08:32:54.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (34 ms) 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (8 ms) 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (7 ms) 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (5 ms) 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (10 ms) 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-11T08:32:54.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (7 ms) 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (4 ms) 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (13 ms) 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (12 ms) 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (10 ms) 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (4 ms) 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (405 ms total) 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-11T08:32:54.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (14451 ms total) 2024-09-11T08:32:54.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-09-11T08:32:54.519 INFO:tasks.workunit.client.0.smithi043.stdout: 50 objects... 2024-09-11T08:32:54.519 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[80820]: added 50 objects 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[80820]: shutting down. 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: starting. 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: listing objects. 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: listed object 0... 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: listed object 25... 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: listed object 50... 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: listed object 75... 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: listed object 100... 2024-09-11T08:32:54.520 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: listed object 125... 2024-09-11T08:32:54.521 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: saw 150 objects 2024-09-11T08:32:54.521 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[80819]: shutting down. 2024-09-11T08:32:54.521 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.521 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.521 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.521 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.521 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-11T08:32:54.521 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******* SUCCESS ********** 2024-09-11T08:32:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: pgmap v76: 868 pgs: 288 creating+peering, 442 unknown, 138 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:32:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:32:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1446711838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1466604476' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:32:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:32:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:32:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: pgmap v76: 868 pgs: 288 creating+peering, 442 unknown, 138 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:32:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:32:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1446711838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1466604476' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:54.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: pgmap v76: 868 pgs: 288 creating+peering, 442 unknown, 138 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1446711838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1466604476' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:32:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:32:55.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:32:55.842 INFO:tasks.workunit.client.0.smithi043.stdout:tchNotify_smithi043-79603-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:55.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.407303+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:55.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.407398+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-79772-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.407530+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-79312-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.407645+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.407964+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.418875+0000 mon.c [INF] from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:51.420881+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:52.985119+0000 mon.c [INF] from='client.? 172.21.15.43:0/1446711838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:53.481276+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:53.486271+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-79622-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:53.490910+0000 mon.c [INF] from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:53.509186+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:55.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:53.598701+0000 mon.b [INF] from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:32:55.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.028892+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:32:56.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:32:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:56.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: pgmap v79: 748 pgs: 76 creating+peering, 40 unknown, 632 active+clean; 612 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 25 KiB/s rd, 148 KiB/s wr, 710 op/s 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: 14.0 deep-scrub starts 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: 14.0 deep-scrub ok 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:56.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:56.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:32:56.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:32:56.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:32:56.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:32:56.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:32:56.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:32:56.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:32:56.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:32:56.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: pgmap v79: 748 pgs: 76 creating+peering, 40 unknown, 632 active+clean; 612 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 25 KiB/s rd, 148 KiB/s wr, 710 op/s 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: 14.0 deep-scrub starts 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: 14.0 deep-scrub ok 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:56.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:32:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:32:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:32:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:32:56.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:32:56.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:32:56.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:32:56.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:32:56.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:32:56.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:32:56.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: pgmap v79: 748 pgs: 76 creating+peering, 40 unknown, 632 active+clean; 612 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 25 KiB/s rd, 148 KiB/s wr, 710 op/s 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: 14.0 deep-scrub starts 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: 14.0 deep-scrub ok 2024-09-11T08:32:56.235 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:32:56.236 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.236 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:56.236 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:56.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.029517+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:32:56.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.030002+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:32:56.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.030468+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:32:56.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.030958+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:32:56.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.031422+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.031884+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.032326+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.032773+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.033222+0000 mon.c [INF] from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.482225+0000 mon.c [INF] from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.484352+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.488310+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-79089-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.488424+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.494891+0000 mon.c [INF] from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.505084+0000 mon.b [INF] from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:56.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.595732+0000 mon.c [INF] from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:56.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.962590+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:56.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:54.963416+0000 mon.c [INF] from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:56.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:53.598959+0000 client.admin [INF] onexx 2024-09-11T08:32:56.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:55.835739+0000 mon.a [INF] from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:56.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:55.839183+0000 mon.a [INF] from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:56.876 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:55.839701+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:32:56.876 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:55.847485+0000 mon.c [INF] from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:56.876 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:55.855626+0000 mon.b [INF] from='client.? 172.21.15.43:0/1994833955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:56.876 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.290417+0000 mon.a [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:32:56.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.292258+0000 mon.a [INF] from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:56.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.292493+0000 mon.a [INF] from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:56.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.292867+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:56.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.294119+0000 mon.a [INF] from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:56.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.294809+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:56.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.295038+0000 mon.c [INF] from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:56.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.295445+0000 mon.a [INF] from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-79529-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:56.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.295797+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:56.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.296033+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:56.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.299889+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: onexx 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: 14.2 deep-scrub starts 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: 14.2 deep-scrub ok 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: 14.8 deep-scrub starts 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: 14.8 deep-scrub ok 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: 14.3 deep-scrub starts 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: 14.3 deep-scrub ok 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1994833955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: 14.4 deep-scrub starts 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: 14.4 deep-scrub ok 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: pgmap v82: 716 pgs: 76 creating+peering, 200 unknown, 440 active+clean; 586 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 523 op/s 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-79529-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: onexx 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: 14.2 deep-scrub starts 2024-09-11T08:32:57.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: 14.2 deep-scrub ok 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: 14.8 deep-scrub starts 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: 14.8 deep-scrub ok 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: 14.3 deep-scrub starts 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: 14.3 deep-scrub ok 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1994833955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: 14.4 deep-scrub starts 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: 14.4 deep-scrub ok 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:57.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: pgmap v82: 716 pgs: 76 creating+peering, 200 unknown, 440 active+clean; 586 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 523 op/s 2024-09-11T08:32:57.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:32:57.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:32:57.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:57.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:57.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.149 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-79529-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: onexx 2024-09-11T08:32:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: 14.2 deep-scrub starts 2024-09-11T08:32:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: 14.2 deep-scrub ok 2024-09-11T08:32:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: 14.8 deep-scrub starts 2024-09-11T08:32:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: 14.8 deep-scrub ok 2024-09-11T08:32:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: 14.3 deep-scrub starts 2024-09-11T08:32:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: 14.3 deep-scrub ok 2024-09-11T08:32:57.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1994833955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: 14.4 deep-scrub starts 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: 14.4 deep-scrub ok 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: pgmap v82: 716 pgs: 76 creating+peering, 200 unknown, 440 active+clean; 586 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 523 op/s 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:57.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-79529-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:57.234 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:59.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.300443+0000 mon.c [INF] from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-pro delete_pools_parallel: process_1_[79948]: starting. 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[79948]: creating pool ceph_test_rados_delete_pools_parallel.smithi043-79912 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[79948]: created object 0... 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[79948]: created object 25... 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[79948]: created object 49... 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[79948]: finishing. 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[79948]: shutting down. 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[79950]: starting. 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[79950]: deleting pool ceph_test_rados_delete_pools_parallel.smithi043-79912 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[79950]: shutting down. 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:32:59.607 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[80899]: starting. 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[80899]: creating pool ceph_test_rados_delete_pools_parallel.smithi043-79912 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[80899]: created object 0... 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[80899]: created object 25... 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[80899]: created object 49... 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[80899]: finishing. 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[80899]: shutting down. 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[80901]: starting. 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[80901]: listing objects. 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[80901]: listed object 0... 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[80901]: listed object 25... 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[80901]: saw 50 objects 2024-09-11T08:32:59.608 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[80901]: shutting down. 2024-09-11T08:32:59.609 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:32:59.609 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[80900]: starting. 2024-09-11T08:32:59.609 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[80900]: deleting pool ceph_test_rados_delete_pools_parallel.smithi043-79912 2024-09-11T08:32:59.609 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[80900]: shutting down. 2024-09-11T08:32:59.609 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:32:59.609 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:32:59.609 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[79932]: starting. 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[79932]: creating pool ceph_test_rados_open_pools_parallel.smithi043-79909 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[79932]: created object 0... 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[79932]: created object 25... 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[79932]: created object 49... 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[79932]: finishing. 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[79932]: shutting down. 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[79933]: starting. 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[79933]: rados_pool_create. 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[79933]: rados_ioctx_create. 2024-09-11T08:32:59.622 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[79933]: shutting down. 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[80902]: starting. 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[80902]: creating pool ceph_test_rados_open_pools_parallel.smithi043-79909 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[80902]: created object 0... 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[80902]: created object 25... 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[80902]: created object 49... 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[80902]: finishing. 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[80902]: shutting down. 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[80903]: starting. 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[80903]: rados_pool_create. 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[80903]: rados_ioctx_create. 2024-09-11T08:32:59.623 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[80903]: shutting down. 2024-09-11T08:32:59.624 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-09-11T08:32:59.624 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-09-11T08:32:59.624 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: 14.1 deep-scrub starts 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: 14.1 deep-scrub ok 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: 14.5 deep-scrub starts 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: 14.5 deep-scrub ok 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-79529-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:32:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-79572-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-79529-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:59.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2818038828' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:59.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: 14.1 deep-scrub starts 2024-09-11T08:32:59.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: 14.1 deep-scrub ok 2024-09-11T08:32:59.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: 14.5 deep-scrub starts 2024-09-11T08:32:59.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: 14.5 deep-scrub ok 2024-09-11T08:32:59.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-79529-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-79572-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-79529-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:59.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2818038828' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: twoxx 2024-09-11T08:32:59.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.901 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.901 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-11T08:32:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:32:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: twoxx 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:59.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:32:59 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:32:59.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: 14.1 deep-scrub starts 2024-09-11T08:32:59.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: 14.1 deep-scrub ok 2024-09-11T08:32:59.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: 14.5 deep-scrub starts 2024-09-11T08:32:59.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: 14.5 deep-scrub ok 2024-09-11T08:32:59.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-79124-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-79529-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-79572-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-79529-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:59.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2818038828' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: twoxx 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:32:59.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:32:59.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:32:59.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:32:59 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:00.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: Running main() from gmock_main.cc 2024-09-11T08:33:00.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-09-11T08:33:00.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-11T08:33:00.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-09-11T08:33:00.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: seed 79142 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (51 ms) 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (7 ms) 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (4 ms) 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (1223 ms) 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-09-11T08:33:00.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (5 ms) 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (4 ms) 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (11 ms) 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (4 ms) 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (5 ms) 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-11T08:33:00.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (7 ms) 2024-09-11T08:33:00.726 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-11T08:33:00.726 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (3 ms) 2024-09-11T08:33:00.726 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-11T08:33:00.726 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (6 ms) 2024-09-11T08:33:00.726 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-11T08:33:00.726 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (16 ms) 2024-09-11T08:33:00.726 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-11T08:33:00.726 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (10 ms) 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (4 ms) 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (4 ms) 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (1374 ms total) 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: 2024-09-11T08:33:00.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-09-11T08:33:00.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-11T08:33:00.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1074 ms) 2024-09-11T08:33:00.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-11T08:33:00.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (13 ms) 2024-09-11T08:33:00.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-11T08:33:00.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-09-11T08:33:00.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-11T08:33:00.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (3 ms) 2024-09-11T08:33:00.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-11T08:33:00.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (4 ms) 2024-09-11T08:33:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: pgmap v85: 548 pgs: 48 creating+peering, 16 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: 14.6 deep-scrub starts 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: 14.6 deep-scrub ok 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: 14.9 deep-scrub starts 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: 14.9 deep-scrub ok 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2818038828' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: 14.7 deep-scrub starts 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: 14.7 deep-scrub ok 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:00.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2010443875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-11T08:33:00.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: pgmap v85: 548 pgs: 48 creating+peering, 16 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:00.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: 14.6 deep-scrub starts 2024-09-11T08:33:00.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: 14.6 deep-scrub ok 2024-09-11T08:33:00.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: 14.9 deep-scrub starts 2024-09-11T08:33:00.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: 14.9 deep-scrub ok 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2818038828' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: 14.7 deep-scrub starts 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: 14.7 deep-scrub ok 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:33:00.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:33:00.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:00.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:00.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:00.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2010443875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-79572-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-79529-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:00.901 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.901 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.901 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: pgmap v87: 580 pgs: 48 creating+peering, 48 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 23 op/s 2024-09-11T08:33:00.901 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-79572-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-79529-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: pgmap v87: 580 pgs: 48 creating+peering, 48 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 23 op/s 2024-09-11T08:33:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: pgmap v85: 548 pgs: 48 creating+peering, 16 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:00.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: 14.6 deep-scrub starts 2024-09-11T08:33:00.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: 14.6 deep-scrub ok 2024-09-11T08:33:00.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: 14.9 deep-scrub starts 2024-09-11T08:33:00.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: 14.9 deep-scrub ok 2024-09-11T08:33:00.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2818038828' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:00.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: 14.7 deep-scrub starts 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: 14.7 deep-scrub ok 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:00.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2010443875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-79572-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-79529-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:33:00.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:33:00.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:00.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:00.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:00.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:00.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:00.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: pgmap v87: 580 pgs: 48 creating+peering, 48 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 23 op/s 2024-09-11T08:33:00.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:01.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN file set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:01.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.300883+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:01.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:56.301870+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:01.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.166718+0000 mon.a [INF] from='client.? 172.21.15.43:0/2818038828' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:01.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.600723+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.600821+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.601149+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.601252+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-79202-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.601736+0000 mon.c [INF] from='client.? 172.21.15.43:0/908601285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.602160+0000 mon.c [INF] from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.602457+0000 mon.b [INF] from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.602495+0000 mon.c [INF] from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.603066+0000 mon.c [INF] from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.603251+0000 mon.c [INF] from='client.? 172.21.15.43:0/2010443875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:01.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.616103+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-11T08:33:02.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.622636+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-79124-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:02.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.623004+0000 mon.a [INF] from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-79572-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:02.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.623223+0000 mon.a [INF] from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-79529-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:02.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.623434+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-79142-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:02.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.639577+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]: dispatch 2024-09-11T08:33:02.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.640010+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:02.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.640380+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:02.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.641651+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:02.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:32:59.642365+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:00.510790+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:00.626327+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:33:02.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:00.626394+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:02.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:00.635524+0000 mon.c [INF] from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:33:02.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.513059+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1884230415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:02.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:33:02.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:33:02.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:33:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:33:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:33:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1884230415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:33:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-79622-1"}]': finished 2024-09-11T08:33:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-4", "overlaypool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:33:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-79089-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:33:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1884230415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:33:02.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:02.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:02.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:33:02.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:33:02.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:02.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:33:03.414 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.513294+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-79112-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.414 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.514178+0000 mon.c [INF] from='client.? 172.21.15.43:0/1884230415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.414 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.516221+0000 mon.b [INF] from='client.? 172.21.15.43:0/717972392' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:33:03.414 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.524330+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-79226-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:03.415 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.524549+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-79202-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:03.415 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.526321+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:33:03.415 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.529895+0000 mon.a [INF] from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.415 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.531519+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.415 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.534795+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:33:03.415 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.617111+0000 mon.b [INF] from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:33:03.415 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:01.617257+0000 client.admin [INF] threexx 2024-09-11T08:33:03.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.408411+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:33:03.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.521266+0000 mon.a [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:03.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.525895+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]': finished 2024-09-11T08:33:03.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.526068+0000 mon.a [INF] from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.526291+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.526414+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-11T08:33:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: threexx 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: pgmap v90: 628 pgs: 48 creating+peering, 128 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 28 op/s 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]': finished 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3726730193' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]: dispatch 2024-09-11T08:33:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:03.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:03.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:33:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: threexx 2024-09-11T08:33:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: pgmap v90: 628 pgs: 48 creating+peering, 128 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 28 op/s 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]': finished 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3726730193' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]: dispatch 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: threexx 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: pgmap v90: 628 pgs: 48 creating+peering, 128 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 28 op/s 2024-09-11T08:33:03.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-4-cache", "mode": "writeback"}]': finished 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-79112-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3726730193' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:33:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:33:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]: dispatch 2024-09-11T08:33:03.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-11T08:33:03.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:03.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]: dispatch 2024-09-11T08:33:03.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:03.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:03.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:03.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-11T08:33:03.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:03.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:03.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]: dispatch 2024-09-11T08:33:03.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:03.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:03.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:04.145 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:33:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:03] "GET /metrics HTTP/1.1" 200 256066 "" "Prometheus/2.43.0" 2024-09-11T08:33:04.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.528553+0000 mon.c [INF] from='client.? 172.21.15.43:0/3665881710' entity ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-11T08:33:04.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (3 ms) 2024-09-11T08:33:04.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-11T08:33:04.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (4 ms) 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (3 ms) 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (4 ms) 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (3 ms) 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-09-11T08:33:04.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (5 ms) 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (2 ms) 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1145 ms total) 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (24725 ms total) 2024-09-11T08:33:04.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-09-11T08:33:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: fourxx 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:04.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]': finished 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-79603-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2"}]: dispatch 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3211730755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:04.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:04.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:04.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pg_num": 4}]: dispatch 2024-09-11T08:33:04.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:04.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: fourxx 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]': finished 2024-09-11T08:33:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-79603-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2"}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3211730755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:04.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:04.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:04.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pg_num": 4}]: dispatch 2024-09-11T08:33:04.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:04.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-11T08:33:04.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1565874508' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:33:04.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: fourxx 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:33:04.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-79603-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2"}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3211730755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3665881710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:04.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pg_num": 4}]: dispatch 2024-09-11T08:33:04.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: Running main() from gmock_main.cc 2024-09-11T08:33:04.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-11T08:33:04.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] Global test environment set-up. 2024-09-11T08:33:04.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-11T08:33:04.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-11T08:33:04.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (5 ms) 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (30 ms) 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (4 ms) 2024-09-11T08:33:04.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.Checksum (2 ms) 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (3 ms) 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (4 ms) 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (3 ms) 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-11T08:33:04.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (2 ms) 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (3 ms) 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RmXattr (10 ms) 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 14 tests from LibRadosIo (77 ms total) 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-11T08:33:04.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1069 ms) 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (5 ms) 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (4 ms) 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (5 ms) 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (7 ms) 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (4 ms) 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-09-11T08:33:04.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4 ms) 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1115 ms total) 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] Global test environment tear-down 2024-09-11T08:33:04.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [==========] 24 tests from 2 test suites ran. (25173 ms total) 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ PASSED ] 24 tests. 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: Running main() from gmock_main.cc 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] Global test environment set-up. 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.Stat (284 ms) 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.StatNS (18 ms) 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-11T08:33:04.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (3 ms) 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStat (305 ms total) 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1059 ms) 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (9 ms) 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (820 ms) 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (10 ms) 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1898 ms total) 2024-09-11T08:33:04.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] Global test environment tear-down 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (25032 ms total) 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ PASSED ] 8 tests. 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: seed 79572 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-11T08:33:04.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (173 ms) 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (101 ms) 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (5 ms) 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (17 ms) 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (297 ms total) 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1053 ms) 2024-09-11T08:33:04.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (831 ms) 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (877 ms) 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (8 ms) 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (2769 ms total) 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (25012 ms total) 2024-09-11T08:33:04.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-11T08:33:05.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: pgmap v93: 596 pgs: 64 creating+peering, 32 unknown, 500 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pg_num": 4}]': finished 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1465818665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:05.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:05.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: pgmap v93: 596 pgs: 64 creating+peering, 32 unknown, 500 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-09-11T08:33:05.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:05.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-11T08:33:05.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:05.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:05.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pg_num": 4}]': finished 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1465818665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:05.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: pgmap v93: 596 pgs: 64 creating+peering, 32 unknown, 500 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:05.799 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-79572-7"}]': finished 2024-09-11T08:33:05.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-79142-22"}]': finished 2024-09-11T08:33:05.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-79529-6"}]': finished 2024-09-11T08:33:05.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:05.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-4", "tierpool": "test-rados-api-smithi043-79312-4-cache"}]': finished 2024-09-11T08:33:05.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-79089-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:05.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-79124-16"}]': finished 2024-09-11T08:33:05.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pg_num": 4}]': finished 2024-09-11T08:33:05.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:33:05.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1465818665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:05.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:05.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-11T08:33:05.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:05.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:05.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:06.469 INFO:tasks.workunit.client.0.smithi043.stdout:='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:06.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.529905+0000 mon.c [INF] from='client.? 172.21.15.43:0/817439427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:06.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.530135+0000 mon.c [INF] from='client.? 172.21.15.43:0/3726730193' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.536630+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-79124-16"}]: dispatch 2024-09-11T08:33:06.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.537484+0000 mon.a [INF] from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.538084+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-79142-22"}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.538335+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.538544+0000 mon.a [INF] from='client.? 172.21.15.43:0/2086583246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-79572-7"}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.540592+0000 mon.a [INF] from='client.? 172.21.15.43:0/1664795473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-79529-6"}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:02.626722+0000 mon.c [INF] from='client.? 172.21.15.43:0/264285592' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:03.022954+0000 mon.a [INF] from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-79305-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:03.024114+0000 mon.a [INF] from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:03.024851+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-4"}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:03.026466+0000 mon.a [INF] from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-11T08:33:03.026861+0000 mon.a [INF] from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (12979 ms) 2024-09-11T08:33:06.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (26459 ms total) 2024-09-11T08:33:06.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: 2024-09-11T08:33:06.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-11T08:33:06.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (26459 ms total) 2024-09-11T08:33:06.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-11T08:33:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-79603-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]: dispatch 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1210123411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: pool 'PoolQuotaPP_smithi043-79112-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]': finished 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-79603-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:06.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]: dispatch 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1210123411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: pool 'PoolQuotaPP_smithi043-79112-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:33:06.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]': finished 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1210123411' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/349665609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "mode": "writeback"}]: dispatch 2024-09-11T08:33:06.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1210123411' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/349665609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "mode": "writeback"}]: dispatch 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-79603-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]: dispatch 2024-09-11T08:33:06.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1210123411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:06.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:06.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: pool 'PoolQuotaPP_smithi043-79112-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-11T08:33:06.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:33:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]': finished 2024-09-11T08:33:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1210123411' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/936099309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:06.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:33:06.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:33:06.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/349665609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:06.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/978013262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:06.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "mode": "writeback"}]: dispatch 2024-09-11T08:33:06.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]: dispatch 2024-09-11T08:33:06.986 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: Running main() from gmock_main.cc 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] Global test environment set-up. 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (179 ms) 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockShared (6 ms) 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1092 ms) 2024-09-11T08:33:07.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1012 ms) 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (2 ms) 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.Unlock (3 ms) 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (4 ms) 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLock (2301 ms total) 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-11T08:33:07.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1091 ms) 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (2 ms) 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1005 ms) 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1005 ms) 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (4 ms) 2024-09-11T08:33:07.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (3 ms) 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3118 ms total) 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] Global test environment tear-down 2024-09-11T08:33:07.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (27686 ms total) 2024-09-11T08:33:07.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-11T08:33:07.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-11T08:33:07.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-11T08:33:07.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-11T08:33:07.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-11T08:33:07.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: seed 79226 2024-09-11T08:33:07.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-11T08:33:07.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (192 ms) 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (87 ms) 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1007 ms) 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1022 ms) 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (4 ms) 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (50 ms) 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-11T08:33:07.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (4 ms) 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (46 ms) 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2412 ms total) 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1089 ms) 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (3 ms) 2024-09-11T08:33:07.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1006 ms) 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (5 ms) 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (3 ms) 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-09-11T08:33:07.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-11T08:33:07.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (2 ms) 2024-09-11T08:33:07.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3118 ms total) 2024-09-11T08:33:07.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: 2024-09-11T08:33:07.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-11T08:33:07.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (27698 ms total) 2024-09-11T08:33:07.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-11T08:33:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: pgmap v96: 560 pgs: 32 creating+peering, 76 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:33:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "mode": "writeback"}]': finished 2024-09-11T08:33:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:07.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:07.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:33:07.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:33:07.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:07.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2294112968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1910463153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: pgmap v96: 560 pgs: 32 creating+peering, 76 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "mode": "writeback"}]': finished 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:33:07.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:07.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2294112968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1910463153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: pgmap v96: 560 pgs: 32 creating+peering, 76 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:33:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]: dispatch 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "mode": "writeback"}]': finished 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-79226-10"}]': finished 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-79089-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-79202-10"}]': finished 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258695569' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-79305-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:33:07.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:07.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2294112968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1910463153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:07.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[29355]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4072528693' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[40592]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4072528693' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-79424-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:08 smithi172 ceph-mon[32987]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:33:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4072528693' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5061 ms) 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (24 ms) 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20020 ms) 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-11T08:33:10.261 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 0 of 20 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: failed to deregister: 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: { 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "epoch": 19, 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "modified": "2024-09-11T08:33:06.188778+0000", 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "services": { 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "laundry": { 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "close-test-pid79776": { 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 19, 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-11T08:33:05.180358+0000", 2024-09-11T08:33:10.262 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 25100, 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/320393102", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-11T08:33:10.263 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493484", 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid79765": { 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 18, 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-11T08:32:53.040830+0000", 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24949, 2024-09-11T08:33:10.264 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/3041567415", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:33:10.265 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493484", 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "rgw": { 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-09-11T08:33:10.266 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "24368": { 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 12, 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-11T08:30:10.230814+0000", 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24368, 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/2012474113", 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_hostname": "smithi043", 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40", 2024-09-11T08:33:10.267 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_type#0": "beast", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "id": "foo.a", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493484", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "num_handles": "1", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:33:10.268 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid": "7", 2024-09-11T08:33:10.269 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_id": "", 2024-09-11T08:33:10.269 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_name": "", 2024-09-11T08:33:10.269 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_id": "c406385e-5cbc-4dfb-a261-e537cf3e190f", 2024-09-11T08:33:10.269 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_name": "default", 2024-09-11T08:33:10.269 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_id": "c3d9ab5b-b0eb-422f-bbe1-4da8fc4a3ae1", 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_na api_watch_notify: Running main() from gmock_main.cc 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify_test_cb 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (280 ms) 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94610944846288 err -107 2024-09-11T08:33:10.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (14 ms) 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94610944848496 err -107 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (10 ms) 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 300647710720 cookie 94610944865344 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (9 ms) 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 300647710721 cookie 94610944865344 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (11 ms) 2024-09-11T08:33:10.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 300647710722 cookie 94610944865344 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (10 ms) 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 300647710721 cookie 94610944897840 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 300647710721 cookie 94610944920064 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (16 ms) 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 300647710722 cookie 94610944897840 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 304942678019 cookie 94610944897840 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3006 ms) 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-11T08:33:10.393 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94610944920064 err -107 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 330712481793 cookie 94610944920064 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5008 ms) 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94610944900288 err -107 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9369 ms total) 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify_test_cb 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1058 ms) 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1058 ms total) 2024-09-11T08:33:10.394 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: 2024-09-11T08:33:10.395 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-11T08:33:10.395 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (30399 ms total) 2024-09-11T08:33:10.395 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-11T08:33:10.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: pgmap v99: 488 pgs: 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[40592]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: pgmap v99: 488 pgs: 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-11T08:33:10.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:10.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:10.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:10.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:10.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:10.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:10.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:10.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:10 smithi043 ceph-mon[29355]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:33:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: pgmap v99: 488 pgs: 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-11T08:33:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]: dispatch 2024-09-11T08:33:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2127565370' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-79603-12"}]': finished 2024-09-11T08:33:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:10.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:10.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:10 smithi172 ceph-mon[32987]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:33:10.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-11T08:33:10.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-11T08:33:10.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-11T08:33:10.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-11T08:33:10.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-11T08:33:10.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-09-11T08:33:10.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1129 ms) 2024-09-11T08:33:10.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (1269 ms) 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (2398 ms total) 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (64 ms) 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4037 ms) 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (18 ms) 2024-09-11T08:33:10.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (10 ms) 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906418736 notify_id 365072220162 notifier_gid 15126 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (8 ms) 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906418736 notify_id 365072220164 notifier_gid 15126 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (16 ms) 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906441824 notify_id 365072220163 notifier_gid 15126 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (9 ms) 2024-09-11T08:33:10.735 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906418736 notify_id 365072220163 notifier_gid 15126 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (17 ms) 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906441792 notify_id 365072220164 notifier_gid 15126 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (7 ms) 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906441792 notify_id 365072220165 notifier_gid 15126 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (8 ms) 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: trying... 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906441792 notify_id 365072220165 notifier_gid 15126 2024-09-11T08:33:10.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: timed out 2024-09-11T08:33:10.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-09-11T08:33:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: pgmap v102: 456 pgs: 64 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 3.2 KiB/s wr, 2 op/s 2024-09-11T08:33:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1503439722' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:33:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3643523617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: pgmap v102: 456 pgs: 64 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 3.2 KiB/s wr, 2 op/s 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1503439722' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3643523617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: pgmap v102: 456 pgs: 64 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 3.2 KiB/s wr, 2 op/s 2024-09-11T08:33:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1503439722' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:33:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3643523617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.047 INFO:tasks.workunit.client.0.smithi043.stdout: ap api_service: [==========] Running 4 tests from 1 test suite. 2024-09-11T08:33:13.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] Global test environment set-up. 2024-09-11T08:33:13.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5070 ms) 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.RegisterLate (31 ms) 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_OK 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-11T08:33:13.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 6 pools, 132 pgs 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 132 active+clean 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: 327ee848-7017-11ef-bcea-c7b262605968 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_WARN 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 20 pool(s) do not have an application enabled 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 26 pools, 772 pgs 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 36.528% pgs unknown 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 44.689% pgs not active 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 345 creating+peering 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 282 unknown 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 145 active+clean 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.StatusFormat (7865 ms) 2024-09-11T08:33:13.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-11T08:33:13.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.Status (20023 ms) 2024-09-11T08:33:13.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] 4 tests from LibRadosService (32989 ms total) 2024-09-11T08:33:13.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-11T08:33:13.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] Global test environment tear-down 2024-09-11T08:33:13.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [==========] 4 tests from 1 test suite ran. (32989 ms total) 2024-09-11T08:33:13.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ PASSED ] 4 tests. 2024-09-11T08:33:13.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:13.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:13.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:13.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:13.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:33:13.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/590315470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366683039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:13.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:13.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:13.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:13.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:13.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:13.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:13.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:33:13.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/590315470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366683039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:13.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:13.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:33:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/590315470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366683039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:14.164 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: pgmap v105: 552 pgs: 160 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:14.164 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:33:14.164 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:14.164 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:14.164 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:33:14.164 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/124994375' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:14.164 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:14.165 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:14.165 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:14.165 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:14.165 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:13] "GET /metrics HTTP/1.1" 200 254311 "" "Prometheus/2.43.0" 2024-09-11T08:33:14.165 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: pgmap v105: 552 pgs: 160 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:14.165 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:33:14.165 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:14.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:14.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:33:14.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/124994375' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:14.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:14.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:14.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:14.166 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: pgmap v105: 552 pgs: 160 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:33:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-79089-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-79495-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:33:14.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/124994375' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:14.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:14.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:14.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:14.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:15.439 INFO:tasks.workunit.client.0.smithi043.stdout:me": "default" 2024-09-11T08:33:15.439 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:15.439 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-11T08:33:15.439 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.439 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.439 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.439 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.439 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 1 of 20 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: failed to deregister: 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: { 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "epoch": 21, 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "modified": "2024-09-11T08:33:12.192232+0000", 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "services": { 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "laundry": { 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "close-test-pid79776": { 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 21, 2024-09-11T08:33:15.440 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-11T08:33:10.305728+0000", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 27208, 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/337201644", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-09-11T08:33:15.441 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493484", 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid79765": { 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 18, 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-11T08:32:53.040830+0000", 2024-09-11T08:33:15.442 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24949, 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/3041567415", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-11T08:33:15.443 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493484", 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "rgw": { 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-09-11T08:33:15.444 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "24368": { 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 12, 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-11T08:30:10.230814+0000", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24368, 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/2012474113", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_hostname": "smithi043", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-11T08:33:15.445 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_type#0": "beast", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "id": "foo.a", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493484", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "num_handles": "1", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid": "7", 2024-09-11T08:33:15.446 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_id": "", 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_name": "", 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_id": "c406385e-5cbc-4dfb-a261-e537cf3e190f", 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_name": "default", 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_id": "c3d9ab5b-b0eb-422f-bbe1-4da8fc4a3ae1", 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_name": "default" 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:15.447 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-09-11T08:33:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:33:15.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:33:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:33:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: pgmap v108: 488 pgs: 25 creating+peering, 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 39 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:33:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3762960364' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: pgmap v108: 488 pgs: 25 creating+peering, 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 39 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3762960364' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:33:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: pgmap v108: 488 pgs: 25 creating+peering, 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 39 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:33:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3762960364' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]': finished 2024-09-11T08:33:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:16 smithi172 ceph-mon[32987]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:33:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2898677025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4081147928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:17.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]': finished 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[29355]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2898677025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4081147928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-7"}]': finished 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[40592]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:33:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2898677025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4081147928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: pgmap v110: 488 pgs: 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 64 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:33:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/101882097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/979244677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: pgmap v110: 488 pgs: 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 64 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:33:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:18.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:18.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:18.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:33:18.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:18.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/101882097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:18.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:18.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/979244677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:18.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:18.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:18.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:18.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:18.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: pgmap v110: 488 pgs: 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 64 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-79089-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/101882097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/979244677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:18.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:18.918 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 2 of 2 api_c_write_operations: Running main() from gmock_main.cc 2024-09-11T08:33:18.918 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (1 ms) 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (1 ms total) 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (14477 ms) 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (7021 ms) 2024-09-11T08:33:18.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3454 ms) 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2546 ms) 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (4654 ms) 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3278 ms) 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3456 ms) 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (38886 ms total) 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: 2024-09-11T08:33:18.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-11T08:33:18.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (38887 ms total) 2024-09-11T08:33:18.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-11T08:33:19.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:19.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:19.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:19.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:33:19.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:19.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:33:19.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:33:19.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79755-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:33:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]: dispatch 2024-09-11T08:33:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:33:19.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:33:19.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:20.882 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: pgmap v113: 488 pgs: 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 64 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:20.882 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]': finished 2024-09-11T08:33:20.882 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:33:20.882 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2557440012' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4"}]: dispatch 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4"}]': finished 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:33:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]: dispatch 2024-09-11T08:33:20.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: pgmap v113: 488 pgs: 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 64 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:20.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]': finished 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2557440012' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4"}]: dispatch 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4"}]': finished 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:33:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]: dispatch 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: pgmap v113: 488 pgs: 43 creating+activating, 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 64 unknown, 375 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-7"}]': finished 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2557440012' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:20.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4"}]: dispatch 2024-09-11T08:33:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-79089-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4"}]': finished 2024-09-11T08:33:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:33:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]: dispatch 2024-09-11T08:33:21.207 INFO:tasks.workunit.client.0.smithi043.stdout:0 2024-09-11T08:33:21.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: failed to deregister: 2024-09-11T08:33:21.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: { 2024-09-11T08:33:21.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "epoch": 23, 2024-09-11T08:33:21.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "modified": "2024-09-11T08:33:17.997571+0000", 2024-09-11T08:33:21.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "services": { 2024-09-11T08:33:21.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "laundry": { 2024-09-11T08:33:21.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "close-test-pid79776": { 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 23, 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-11T08:33:16.673443+0000", 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 20787, 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/3928643364", 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:33:21.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493484", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-09-11T08:33:21.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:21.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-11T08:33:21.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:21.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:21.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:21.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "rgw": { 2024-09-11T08:33:21.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-09-11T08:33:21.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-09-11T08:33:21.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "24368": { 2024-09-11T08:33:21.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 12, 2024-09-11T08:33:21.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-11T08:30:10.230814+0000", 2024-09-11T08:33:21.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24368, 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/2012474113", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_hostname": "smithi043", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:33:21.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:33:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/824657852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: pgmap v116: 456 pgs: 64 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 538 B/s wr, 2 op/s 2024-09-11T08:33:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:22.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:22.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:22.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:22.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]': finished 2024-09-11T08:33:22.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:22.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:22.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:33:22.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/824657852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: pgmap v116: 456 pgs: 64 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 538 B/s wr, 2 op/s 2024-09-11T08:33:22.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:22.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:22.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:22.402 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:22.402 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]': finished 2024-09-11T08:33:22.402 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:22.402 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:22.402 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:33:22.402 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/824657852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: pgmap v116: 456 pgs: 64 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 538 B/s wr, 2 op/s 2024-09-11T08:33:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6"}]': finished 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-79112-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:22.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_i_watch_notify_pp: flushed 2024-09-11T08:33:23.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3007 ms) 2024-09-11T08:33:23.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-11T08:33:23.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: trying... 2024-09-11T08:33:23.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906441760 notify_id 373662154758 notifier_gid 15126 2024-09-11T08:33:23.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: timed out 2024-09-11T08:33:23.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushed 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3009 ms) 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: List watches 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906441760 notify_id 386547056646 notifier_gid 15126 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 done 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: watch_check 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: unwatch2 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: done 2024-09-11T08:33:23.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3021 ms) 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: List watches 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93825906441760 notify_id 399431958535 notifier_gid 15126 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 done 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: watch_check 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: unwatch2 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: done 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3020 ms) 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16251 ms total) 2024-09-11T08:33:23.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: 2024-09-11T08:33:23.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-11T08:33:23.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (43470 ms total) 2024-09-11T08:33:23.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-11T08:33:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2031821521' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: pgmap v118: 424 pgs: 64 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 2 op/s 2024-09-11T08:33:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/311832761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4006008465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2031821521' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: pgmap v118: 424 pgs: 64 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 2 op/s 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/311832761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4006008465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:23.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2031821521' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: pgmap v118: 424 pgs: 64 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 2 op/s 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4017223498' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-79622-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/311832761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4006008465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:33:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:23] "GET /metrics HTTP/1.1" 200 254311 "" "Prometheus/2.43.0" 2024-09-11T08:33:24.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:24.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:24.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:24.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:24.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:24.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:33:24.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3054943475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:33:24.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3054943475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:24.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:33:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:33:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-79424-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-79089-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3054943475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:33:24.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:33:24.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:33:24.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:33:24.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:33:24.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:33:24.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:33:24.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:33:24.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:33:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: pgmap v121: 460 pgs: 40 unknown, 69 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:33:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:33:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:33:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:33:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:33:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]': finished 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]: dispatch 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]: dispatch 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: 14.0 deep-scrub starts 2024-09-11T08:33:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: 14.0 deep-scrub ok 2024-09-11T08:33:25.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:25.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: pgmap v121: 460 pgs: 40 unknown, 69 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:25.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:33:25.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:33:25.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:33:25.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:33:25.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:33:25.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:33:25.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:33:25.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:33:25.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:33:25.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:33:25.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:33:25.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]': finished 2024-09-11T08:33:25.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:25.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:33:25.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]: dispatch 2024-09-11T08:33:25.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]: dispatch 2024-09-11T08:33:25.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: 14.0 deep-scrub starts 2024-09-11T08:33:25.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: 14.0 deep-scrub ok 2024-09-11T08:33:25.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:33:25.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: pgmap v121: 460 pgs: 40 unknown, 69 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-09-11T08:33:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-9"}]': finished 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-79112-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]: dispatch 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]: dispatch 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: 14.0 deep-scrub starts 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: 14.0 deep-scrub ok 2024-09-11T08:33:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: 14.1 deep-scrub starts 2024-09-11T08:33:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: 14.1 deep-scrub ok 2024-09-11T08:33:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]': finished 2024-09-11T08:33:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:33:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3226548094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:26.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:26.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:26.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:26.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:26.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:26.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: 14.1 deep-scrub starts 2024-09-11T08:33:26.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: 14.1 deep-scrub ok 2024-09-11T08:33:26.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:26.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]': finished 2024-09-11T08:33:26.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:33:26.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3226548094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:26.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:26.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:26.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:26.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:26.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:26.671 INFO:tasks.workunit.client.0.smithi043.stdout:service_pp: "frontend_config#0": "beast port=80", 2024-09-11T08:33:26.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_type#0": "beast", 2024-09-11T08:33:26.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-11T08:33:26.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "id": "foo.a", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493484", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "num_handles": "1", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid": "7", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_id": "", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_name": "", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_id": "c406385e-5cbc-4dfb-a261-e537cf3e190f", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_name": "default", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_id": "c3d9ab5b-b0eb-422f-bbe1-4da8fc4a3ae1", 2024-09-11T08:33:26.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_name": "default" 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 3 of 20 2024-09-11T08:33:26.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (21502 ms) 2024-09-11T08:33:26.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (46607 ms total) 2024-09-11T08:33:26.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-09-11T08:33:26.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-11T08:33:26.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (46608 ms total) 2024-09-11T08:33:26.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-11T08:33:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: 14.1 deep-scrub starts 2024-09-11T08:33:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: 14.1 deep-scrub ok 2024-09-11T08:33:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-9", "mode": "writeback"}]': finished 2024-09-11T08:33:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:33:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3226548094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:26.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:26.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:28.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: pgmap v124: 428 pgs: 40 unknown, 37 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:28.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2278398200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/190225435' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]': finished 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: pgmap v124: 428 pgs: 40 unknown, 37 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2278398200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/190225435' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:28.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:28.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]': finished 2024-09-11T08:33:28.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:33:28.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: pgmap v124: 428 pgs: 40 unknown, 37 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-79089-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2278398200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]: dispatch 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/190225435' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-79112-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-9"}]': finished 2024-09-11T08:33:28.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:33:28.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:28.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:29.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:29.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: 14.2 deep-scrub starts 2024-09-11T08:33:29.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: 14.2 deep-scrub ok 2024-09-11T08:33:29.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:29.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: 14.8 deep-scrub starts 2024-09-11T08:33:29.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: 14.8 deep-scrub ok 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1874264462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: 14.2 deep-scrub starts 2024-09-11T08:33:29.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: 14.2 deep-scrub ok 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: 14.8 deep-scrub starts 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: 14.8 deep-scrub ok 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1874264462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:29.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:29.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:33:29.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: 14.2 deep-scrub starts 2024-09-11T08:33:29.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: 14.2 deep-scrub ok 2024-09-11T08:33:29.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:29.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: 14.8 deep-scrub starts 2024-09-11T08:33:29.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: 14.8 deep-scrub ok 2024-09-11T08:33:29.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:29.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:29.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79395-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:29.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:33:29.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1874264462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:29.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:30.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: pgmap v127: 428 pgs: 72 unknown, 5 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: 14.3 deep-scrub starts 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: 14.3 deep-scrub ok 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2891604129' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:33:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: pgmap v127: 428 pgs: 72 unknown, 5 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: 14.3 deep-scrub starts 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: 14.3 deep-scrub ok 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2891604129' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:33:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: pgmap v127: 428 pgs: 72 unknown, 5 creating+peering, 351 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:30.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: 14.3 deep-scrub starts 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: 14.3 deep-scrub ok 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-79089-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2891604129' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:33:30.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] Global test environment set-up. 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-11T08:33:30.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-11T08:32:39.980+0000 7f4ed36d5600 -1 failed for service _ceph-mon._tcp 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-11T08:32:39.981+0000 7f4ed36d5600 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (78 ms) 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5032 ms) 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5110 ms total) 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55efe45bc2d0 2024-09-11T08:33:30.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55efe45c0ad0 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 2 aios 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55efe45bc2d0 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55efe45c0ad0 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: done. 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (14592 ms) 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (14592 ms total) 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-11T08:33:30.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.Exec (60 ms) 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (22 ms) 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (4 ms) 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4909 ms) 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-11T08:33:30.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-11T08:33:30.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (21987 ms) 2024-09-11T08:33:30.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 8 tests from LibRadosMisc (26982 ms total) 2024-09-11T08:33:30.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-11T08:33:30.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] Global test environment tear-down 2024-09-11T08:33:30.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (50622 ms total) 2024-09-11T08:33:30.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-11T08:33:31.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[40592]: pgmap v130: 492 pgs: 96 unknown, 32 creating+peering, 11 creating+activating, 1 active+clean+snaptrim, 352 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-09-11T08:33:31.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:31.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]': finished 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[40592]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[29355]: pgmap v130: 492 pgs: 96 unknown, 32 creating+peering, 11 creating+activating, 1 active+clean+snaptrim, 352 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]': finished 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[29355]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:33:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:31.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:31.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:31 smithi172 ceph-mon[32987]: pgmap v130: 492 pgs: 96 unknown, 32 creating+peering, 11 creating+activating, 1 active+clean+snaptrim, 352 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-09-11T08:33:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache", "force_nonempty":""}]': finished 2024-09-11T08:33:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-79112-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:31 smithi172 ceph-mon[32987]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:33:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:31.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:32.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:32.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:32.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:33:32.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]: dispatch 2024-09-11T08:33:32.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2683110950' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:32.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2683110950' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:32.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:32.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:32.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79395-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:32.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:33:32.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2742994455' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]: dispatch 2024-09-11T08:33:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2683110950' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]: dispatch 2024-09-11T08:33:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:32.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: pgmap v133: 428 pgs: 96 unknown, 11 creating+activating, 1 active+clean+snaptrim, 320 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 5 op/s 2024-09-11T08:33:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]': finished 2024-09-11T08:33:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:33:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:33.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:33.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1314064243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:33.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:33.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:33.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: 14.5 deep-scrub starts 2024-09-11T08:33:33.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[40592]: 14.5 deep-scrub ok 2024-09-11T08:33:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: pgmap v133: 428 pgs: 96 unknown, 11 creating+activating, 1 active+clean+snaptrim, 320 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 5 op/s 2024-09-11T08:33:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]': finished 2024-09-11T08:33:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:33:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:33.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1314064243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:33.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:33.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:33.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: 14.5 deep-scrub starts 2024-09-11T08:33:33.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-mon[29355]: 14.5 deep-scrub ok 2024-09-11T08:33:33.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: pgmap v133: 428 pgs: 96 unknown, 11 creating+activating, 1 active+clean+snaptrim, 320 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 5 op/s 2024-09-11T08:33:33.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:33.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79395-10", "tierpool":"test-rados-api-smithi043-79395-10-cache"}]': finished 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-79089-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1314064243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: 14.5 deep-scrub starts 2024-09-11T08:33:33.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:33 smithi172 ceph-mon[32987]: 14.5 deep-scrub ok 2024-09-11T08:33:34.139 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:33:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:33] "GET /metrics HTTP/1.1" 200 244837 "" "Prometheus/2.43.0" 2024-09-11T08:33:34.148 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: Running main() from gmock_main.cc 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] Global test environment set-up. 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolList (12587 ms) 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (7107 ms) 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3929 ms) 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3001 ms) 2024-09-11T08:33:34.149 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3207 ms) 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5722 ms) 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5430 ms) 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5540 ms) 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7714 ms) 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] 9 tests from LibRadosPools (54237 ms total) 2024-09-11T08:33:34.150 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: 2024-09-11T08:33:34.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] Global test environment tear-down 2024-09-11T08:33:34.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (54237 ms total) 2024-09-11T08:33:34.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-11T08:33:34.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]': finished 2024-09-11T08:33:34.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:34.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[40592]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:33:34.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]: dispatch 2024-09-11T08:33:34.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]: dispatch 2024-09-11T08:33:34.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:34.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]': finished 2024-09-11T08:33:34.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:34.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[29355]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:33:34.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]: dispatch 2024-09-11T08:33:34.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]: dispatch 2024-09-11T08:33:34.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:34.485 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-11"}]': finished 2024-09-11T08:33:34.486 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:34.486 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:34 smithi172 ceph-mon[32987]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:33:34.486 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]: dispatch 2024-09-11T08:33:34.486 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]: dispatch 2024-09-11T08:33:34.486 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: pgmap v136: 396 pgs: 6 creating+activating, 1 active+clean+snaptrim, 389 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 257 KiB/s wr, 3 op/s 2024-09-11T08:33:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: 14.4 deep-scrub starts 2024-09-11T08:33:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: 14.4 deep-scrub ok 2024-09-11T08:33:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]': finished 2024-09-11T08:33:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1998285948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-79089-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: pgmap v136: 396 pgs: 6 creating+activating, 1 active+clean+snaptrim, 389 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 257 KiB/s wr, 3 op/s 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: 14.4 deep-scrub starts 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: 14.4 deep-scrub ok 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]': finished 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:33:35.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:35.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:35.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1998285948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-79089-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:35.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:35.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: pgmap v136: 396 pgs: 6 creating+activating, 1 active+clean+snaptrim, 389 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 257 KiB/s wr, 3 op/s 2024-09-11T08:33:35.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: 14.4 deep-scrub starts 2024-09-11T08:33:35.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: 14.4 deep-scrub ok 2024-09-11T08:33:35.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:35.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-11", "mode": "writeback"}]': finished 2024-09-11T08:33:35.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:33:35.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:35.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:35.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1998285948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-79089-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:35.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:35.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:33:35.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:33:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:33:36.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:36.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1998285948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-79089-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/957209823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: 14.6 deep-scrub starts 2024-09-11T08:33:36.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:36 smithi172 ceph-mon[32987]: 14.6 deep-scrub ok 2024-09-11T08:33:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1998285948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-79089-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:33:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/957209823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: 14.6 deep-scrub starts 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[40592]: 14.6 deep-scrub ok 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:36.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1998285948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-79089-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4254493295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]: dispatch 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/957209823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: 14.6 deep-scrub starts 2024-09-11T08:33:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:36 smithi043 ceph-mon[29355]: 14.6 deep-scrub ok 2024-09-11T08:33:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: pgmap v139: 388 pgs: 64 unknown, 1 active+clean+snaptrim, 323 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 257 KiB/s wr, 3 op/s 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:37.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:37.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:37.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]': finished 2024-09-11T08:33:37.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:37.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:33:37.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4167296964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:37.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:37.586 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:37.586 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: pgmap v139: 388 pgs: 64 unknown, 1 active+clean+snaptrim, 323 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 257 KiB/s wr, 3 op/s 2024-09-11T08:33:37.586 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:37.586 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:37.586 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:37.586 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]': finished 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:37.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:33:37.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4167296964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:37.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:37.588 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:37.588 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: pgmap v139: 388 pgs: 64 unknown, 1 active+clean+snaptrim, 323 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 257 KiB/s wr, 3 op/s 2024-09-11T08:33:37.588 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-79424-9"}]': finished 2024-09-11T08:33:37.588 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-79112-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:37.588 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:33:37.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:33:37.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:37.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]: dispatch 2024-09-11T08:33:37.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:37.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:33:37.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:37.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:33:37.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-11"}]': finished 2024-09-11T08:33:37.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:33:37.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:33:37.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:37.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4167296964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:37.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:39.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:39.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:40.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: pgmap v142: 356 pgs: 32 unknown, 1 active+clean+snaptrim, 323 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:40.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:40.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:40.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:33:40.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1472790520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-79112-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1472790520' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-79112-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: pgmap v142: 356 pgs: 32 unknown, 1 active+clean+snaptrim, 323 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:33:40.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1472790520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-79112-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1472790520' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-79112-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:33:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: pgmap v142: 356 pgs: 32 unknown, 1 active+clean+snaptrim, 323 active+clean; 218 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-79089-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:33:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1472790520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-79112-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-79424-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1472790520' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-79112-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:33:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[40592]: pgmap v145: 364 pgs: 25 creating+peering, 47 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:33:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[40592]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:33:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1445874383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:41.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[29355]: pgmap v145: 364 pgs: 25 creating+peering, 47 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:33:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[29355]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:33:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1445874383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:41 smithi172 ceph-mon[32987]: pgmap v145: 364 pgs: 25 creating+peering, 47 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:33:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:41.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:41 smithi172 ceph-mon[32987]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:33:41.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1445874383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:41.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:41.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:33:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2971283314' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: 14.9 deep-scrub starts 2024-09-11T08:33:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: 14.9 deep-scrub ok 2024-09-11T08:33:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:42 smithi172 ceph-mon[32987]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:33:42.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:42.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:33:42.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2971283314' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:42.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: 14.9 deep-scrub starts 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: 14.9 deep-scrub ok 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[40592]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-79089-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2971283314' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:42.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: 14.9 deep-scrub starts 2024-09-11T08:33:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: 14.9 deep-scrub ok 2024-09-11T08:33:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-79112-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:42.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:42 smithi043 ceph-mon[29355]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:33:43.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:43] "GET /metrics HTTP/1.1" 200 240190 "" "Prometheus/2.43.0" 2024-09-11T08:33:43.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[40592]: pgmap v148: 396 pgs: 104 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[40592]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4216768065' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[29355]: pgmap v148: 396 pgs: 104 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[29355]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:33:43.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4216768065' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:43.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:43.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:43 smithi172 ceph-mon[32987]: pgmap v148: 396 pgs: 104 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:33:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:43 smithi172 ceph-mon[32987]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:33:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4216768065' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-mon[32987]: pgmap v151: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 84 KiB/s wr, 82 op/s 2024-09-11T08:33:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-mon[32987]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:33:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3559617967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:45.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[40592]: pgmap v151: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 84 KiB/s wr, 82 op/s 2024-09-11T08:33:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[40592]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:33:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3559617967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[29355]: pgmap v151: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 84 KiB/s wr, 82 op/s 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-79089-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[29355]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3559617967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:33:45.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:33:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:33:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:33:46.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:46.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:46.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:46 smithi172 ceph-mon[32987]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:33:46.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:33:46.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:33:46.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:46 smithi172 ceph-mon[32987]: 14.7 deep-scrub starts 2024-09-11T08:33:46.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:46 smithi172 ceph-mon[32987]: 14.7 deep-scrub ok 2024-09-11T08:33:46.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[29355]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:33:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:33:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:33:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[29355]: 14.7 deep-scrub starts 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[29355]: 14.7 deep-scrub ok 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-79112-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[40592]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[40592]: 14.7 deep-scrub starts 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[40592]: 14.7 deep-scrub ok 2024-09-11T08:33:46.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:47.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:47.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: pgmap v154: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 84 KiB/s wr, 82 op/s 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]': finished 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]: dispatch 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]: dispatch 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4173267283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]': finished 2024-09-11T08:33:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:33:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2225423571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:47.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: pgmap v154: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 84 KiB/s wr, 82 op/s 2024-09-11T08:33:47.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]': finished 2024-09-11T08:33:47.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]: dispatch 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]: dispatch 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4173267283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]': finished 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:33:47.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2225423571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.398 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: pgmap v154: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 84 KiB/s wr, 82 op/s 2024-09-11T08:33:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-13"}]': finished 2024-09-11T08:33:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:33:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]: dispatch 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]: dispatch 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4173267283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-13", "mode": "writeback"}]': finished 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-79089-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2225423571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:48.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: 97.2 scrub starts 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: 97.2 scrub ok 2024-09-11T08:33:48.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: 97.7 scrub starts 2024-09-11T08:33:48.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: 97.7 scrub ok 2024-09-11T08:33:48.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:48.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:33:48.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-11T08:33:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: 97.2 scrub starts 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: 97.2 scrub ok 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: 97.7 scrub starts 2024-09-11T08:33:48.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: 97.7 scrub ok 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-11T08:33:48.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: 97.2 scrub starts 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: 97.2 scrub ok 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: 97.7 scrub starts 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: 97.7 scrub ok 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-79112-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:33:48.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-11T08:33:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-11T08:33:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: pgmap v157: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: 97.8 scrub starts 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: 97.8 scrub ok 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:33:49.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4002151168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-79089-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:49.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-11T08:33:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-11T08:33:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-11T08:33:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: pgmap v157: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: 97.8 scrub starts 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: 97.8 scrub ok 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:33:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4002151168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-79089-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-11T08:33:49.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-11T08:33:49.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: pgmap v157: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:49.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:49.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: 97.8 scrub starts 2024-09-11T08:33:49.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: 97.8 scrub ok 2024-09-11T08:33:49.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:49.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:33:49.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4002151168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-79089-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:49.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]: dispatch 2024-09-11T08:33:50.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (5442 ms) 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (4658 ms) 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-11T08:33:50.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2021 ms) 2024-09-11T08:33:50.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-11T08:33:50.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2004 ms) 2024-09-11T08:33:50.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (14125 ms total) 2024-09-11T08:33:50.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-09-11T08:33:50.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-09-11T08:33:50.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-11T08:33:50.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5526 ms) 2024-09-11T08:33:50.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-11T08:33:50.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4684 ms) 2024-09-11T08:33:50.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (10210 ms total) 2024-09-11T08:33:50.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-09-11T08:33:50.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-11T08:33:50.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-11T08:33:50.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3008 ms) 2024-09-11T08:33:50.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-11T08:33:50.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1713 ms) 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (3385 ms) 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1606 ms) 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9712 ms total) 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-11T08:33:50.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4020 ms) 2024-09-11T08:33:50.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-11T08:33:50.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3999 ms) 2024-09-11T08:33:50.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8019 ms total) 2024-09-11T08:33:50.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-09-11T08:33:50.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-11T08:33:50.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (70243 ms total) 2024-09-11T08:33:50.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-09-11T08:33:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4002151168' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-79089-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:50 smithi172 ceph-mon[32987]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:33:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3884366059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4002151168' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-79089-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[40592]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:33:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3884366059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:50.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4002151168' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-79089-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4221257137' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-79424-14"}]': finished 2024-09-11T08:33:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[29355]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:33:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3884366059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:50.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:51.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: 97.3 scrub starts 2024-09-11T08:33:51.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: 97.3 scrub ok 2024-09-11T08:33:51.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: 97.0 scrub starts 2024-09-11T08:33:51.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: 97.0 scrub ok 2024-09-11T08:33:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: pgmap v160: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 120 op/s 2024-09-11T08:33:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: 97.5 scrub starts 2024-09-11T08:33:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: 97.5 scrub ok 2024-09-11T08:33:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:51 smithi172 ceph-mon[32987]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:33:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: 97.3 scrub starts 2024-09-11T08:33:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: 97.3 scrub ok 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: 97.0 scrub starts 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: 97.0 scrub ok 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: pgmap v160: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 120 op/s 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: 97.5 scrub starts 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: 97.5 scrub ok 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[29355]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: 97.3 scrub starts 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: 97.3 scrub ok 2024-09-11T08:33:51.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: 97.0 scrub starts 2024-09-11T08:33:51.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: 97.0 scrub ok 2024-09-11T08:33:51.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: pgmap v160: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 120 op/s 2024-09-11T08:33:51.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: 97.5 scrub starts 2024-09-11T08:33:51.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: 97.5 scrub ok 2024-09-11T08:33:51.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:51.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-79112-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:51.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:51 smithi043 ceph-mon[40592]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:33:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[40592]: 97.9 scrub starts 2024-09-11T08:33:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[40592]: 97.9 scrub ok 2024-09-11T08:33:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[40592]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:33:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3021095128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[29355]: 97.9 scrub starts 2024-09-11T08:33:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[29355]: 97.9 scrub ok 2024-09-11T08:33:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[29355]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:33:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3021095128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:52 smithi172 ceph-mon[32987]: 97.9 scrub starts 2024-09-11T08:33:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:52 smithi172 ceph-mon[32987]: 97.9 scrub ok 2024-09-11T08:33:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:52 smithi172 ceph-mon[32987]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:33:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3021095128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:53.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:53] "GET /metrics HTTP/1.1" 200 240190 "" "Prometheus/2.43.0" 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: 97.4 scrub starts 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: 97.4 scrub ok 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: pgmap v163: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 120 op/s 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: 97.6 scrub starts 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: 97.6 scrub ok 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1991613911' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-79112-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: 97.4 scrub starts 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: 97.4 scrub ok 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: pgmap v163: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 120 op/s 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: 97.6 scrub starts 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: 97.6 scrub ok 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1991613911' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-79112-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:53.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: 97.4 scrub starts 2024-09-11T08:33:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: 97.4 scrub ok 2024-09-11T08:33:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: pgmap v163: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 120 op/s 2024-09-11T08:33:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: 97.6 scrub starts 2024-09-11T08:33:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: 97.6 scrub ok 2024-09-11T08:33:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-79089-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:33:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1991613911' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-79112-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-mon[32987]: pgmap v166: 388 pgs: 32 creating+peering, 5 creating+activating, 351 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:33:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1991613911' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-79112-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-mon[32987]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:33:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:33:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[29355]: pgmap v166: 388 pgs: 32 creating+peering, 5 creating+activating, 351 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:33:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1991613911' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-79112-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[29355]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:33:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[40592]: pgmap v166: 388 pgs: 32 creating+peering, 5 creating+activating, 351 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1991613911' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-79112-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[40592]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:33:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:33:55.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:33:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:33:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:33:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3987976142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79089-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2"}]: dispatch 2024-09-11T08:33:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3083216668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: 97.1 scrub starts 2024-09-11T08:33:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: 97.1 scrub ok 2024-09-11T08:33:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:33:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3987976142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79089-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2"}]: dispatch 2024-09-11T08:33:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3083216668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: 97.1 scrub starts 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: 97.1 scrub ok 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3987976142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79089-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2"}]: dispatch 2024-09-11T08:33:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3083216668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:56.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: 97.1 scrub starts 2024-09-11T08:33:56.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: 97.1 scrub ok 2024-09-11T08:33:56.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: pgmap v169: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3987976142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79089-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:33:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/427868798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: pgmap v169: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3987976142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79089-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/427868798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: pgmap v169: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3987976142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79089-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-79335-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79495-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/427868798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:33:57.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:33:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:58 smithi172 ceph-mon[32987]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:33:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:58 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:58 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:33:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[40592]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:33:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[29355]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:33:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: pgmap v172: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:33:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:33:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4002413670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:33:59 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: pgmap v172: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:33:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:33:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4002413670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:33:59.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: pgmap v172: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:33:59.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:33:59.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:33:59.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4002413670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:59.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3636654961' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:59.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:33:59.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:33:59.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:33:59.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:33:59 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: seed 79335 2024-09-11T08:34:00.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (2 ms) 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (266 ms) 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (23 ms) 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (12 ms) 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (12 ms) 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (3 ms) 2024-09-11T08:34:00.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (6 ms) 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (14 ms) 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (7 ms) 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-11T08:34:00.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (9 ms) 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1435 ms) 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (477 ms) 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: done waiting, doing copies 2024-09-11T08:34:00.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: done waiting 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60704 ms) 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (2 ms) 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4804 ms) 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-11T08:34:00.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-11T08:34:00.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-11T08:34:00.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-11T08:34:00.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1387916250' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:00.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:00.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:00.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:00.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:00.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:00 smithi172 ceph-mon[32987]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:34:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:34:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:34:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1387916250' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:00.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:00.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[29355]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:34:00.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-79089-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:00.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-79335-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:34:00.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:34:00.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1387916250' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-79112-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:00.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:00.649 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:00 smithi043 ceph-mon[40592]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:34:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:01 smithi172 ceph-mon[32987]: pgmap v175: 356 pgs: 15 creating+peering, 49 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:34:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79335-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[40592]: pgmap v175: 356 pgs: 15 creating+peering, 49 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:34:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79335-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[29355]: pgmap v175: 356 pgs: 15 creating+peering, 49 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:34:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79335-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79335-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:34:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1674935091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79335-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: pgmap v178: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3593267656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79335-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1674935091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79335-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: pgmap v178: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3593267656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:03.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79335-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1674935091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79335-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: pgmap v178: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-79495-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-79089-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:34:03.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3593267656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:03.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:34:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:03] "GET /metrics HTTP/1.1" 200 238045 "" "Prometheus/2.43.0" 2024-09-11T08:34:04.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79335-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:04.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:04.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:04 smithi172 ceph-mon[32987]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:34:04.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79335-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:04 smithi043 ceph-mon[40592]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:34:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:04.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79335-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:04.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-79112-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:04.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:04 smithi043 ceph-mon[29355]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:34:04.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[40592]: pgmap v181: 308 pgs: 15 creating+activating, 28 creating+peering, 4 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[40592]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:34:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3368749018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:05.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[29355]: pgmap v181: 308 pgs: 15 creating+activating, 28 creating+peering, 4 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:05.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:05.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[29355]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:34:05.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3368749018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:05.583 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:34:05.584 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-mon[32987]: pgmap v181: 308 pgs: 15 creating+activating, 28 creating+peering, 4 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:05.584 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:05.584 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-mon[32987]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:34:05.584 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3368749018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.584 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.584 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:05.584 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:05.985 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:34:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:34:06.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:34:06.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/366092460' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[29355]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:34:06.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/366092460' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:06.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:06 smithi043 ceph-mon[40592]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:34:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-79089-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:34:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/366092460' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-79112-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79335-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:06 smithi172 ceph-mon[32987]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:34:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[40592]: pgmap v184: 372 pgs: 3 creating+activating, 8 creating+peering, 100 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[40592]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:34:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1375346442' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:07.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[29355]: pgmap v184: 372 pgs: 3 creating+activating, 8 creating+peering, 100 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:07.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:07.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[29355]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:34:07.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1375346442' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:07.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:07 smithi172 ceph-mon[32987]: pgmap v184: 372 pgs: 3 creating+activating, 8 creating+peering, 100 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:07 smithi172 ceph-mon[32987]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:34:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1375346442' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:08.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:08.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:08.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[40592]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:34:08.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:08.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2826044546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:08.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:08.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:08.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:08.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[29355]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:34:08.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:08.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2826044546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:08.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-79089-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:08 smithi172 ceph-mon[32987]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:34:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:08.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2826044546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:08.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:09.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[29355]: pgmap v187: 308 pgs: 3 creating+activating, 8 creating+peering, 36 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:09.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:09.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:09.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:09.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[29355]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:34:09.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:09.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[40592]: pgmap v187: 308 pgs: 3 creating+activating, 8 creating+peering, 36 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:09.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:09.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:09.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:09.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[40592]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:34:09.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:09 smithi172 ceph-mon[32987]: pgmap v187: 308 pgs: 3 creating+activating, 8 creating+peering, 36 unknown, 261 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-79112-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:09 smithi172 ceph-mon[32987]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:34:09.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]: dispatch 2024-09-11T08:34:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:10.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:10 smithi172 ceph-mon[32987]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:34:10.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3319578054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:10.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[40592]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:34:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3319578054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3819632033' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79335-24"}]': finished 2024-09-11T08:34:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[29355]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:34:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3319578054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: pgmap v190: 300 pgs: 13 creating+peering, 19 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:34:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:34:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4197133867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79112-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2561494397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: pgmap v190: 300 pgs: 13 creating+peering, 19 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:34:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4197133867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79112-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2561494397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: pgmap v190: 300 pgs: 13 creating+peering, 19 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:34:12.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:12.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:12.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-79089-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:12.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:34:12.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4197133867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79112-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:12.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2561494397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:12.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:12.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: pgmap v193: 364 pgs: 96 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:34:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4197133867' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79112-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: pgmap v193: 364 pgs: 96 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:34:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4197133867' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79112-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: pgmap v193: 364 pgs: 96 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4197133867' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-79112-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-79335-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:34:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:13] "GET /metrics HTTP/1.1" 200 235656 "" "Prometheus/2.43.0" 2024-09-11T08:34:14.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots_pp: Running main() from gmock_main.cc 2024-09-11T08:34:14.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-09-11T08:34:14.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-11T08:34:14.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-11T08:34:14.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: seed 79495 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (3076 ms) 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4240 ms) 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (3794 ms) 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2452 ms) 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3757 ms) 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (17319 ms total) 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-11T08:34:14.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4624 ms) 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (6634 ms) 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6103 ms) 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3623 ms) 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2050 ms) 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-11T08:34:14.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi043-79495-7 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17951 ms) 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (40985 ms total) 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (10 ms) 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5059 ms) 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5069 ms total) 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-11T08:34:14.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3011 ms) 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2005 ms) 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2001 ms) 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2008 ms) 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9025 ms total) 2024-09-11T08:34:14.165 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-11T08:34:14.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:14.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:34:14.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:14.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:14.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:14.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:14.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:14.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-79089-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:34:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4165634622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]: dispatch 2024-09-11T08:34:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-79495-15"}]': finished 2024-09-11T08:34:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:34:15.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:34:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: pgmap v196: 260 pgs: 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:34:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1165077276' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-79335-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/285639058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-79112-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1165077276' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-79335-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/285639058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-79112-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:15.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: pgmap v196: 260 pgs: 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1165077276' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-79335-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/285639058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-79112-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1165077276' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-79335-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/285639058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-79112-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: pgmap v196: 260 pgs: 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1165077276' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-79335-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/285639058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-79112-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-79089-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1165077276' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-79335-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/285639058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-79112-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:15.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:15.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:15.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:15.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:15.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:15.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: pgmap v199: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:34:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: pgmap v199: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:17.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:17.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:17.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: pgmap v199: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:34:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:34:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:17.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:18.520 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:18.520 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:34:18.520 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:34:18.520 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1145919487' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/60696477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-79335-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/60696477' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-79335-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:34:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1145919487' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/60696477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-79335-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/60696477' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-79335-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:18.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:18.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:18.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:34:18.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:18.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1145919487' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/60696477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-79335-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/60696477' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-79335-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-79112-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:34:18.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:19.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_p api_aio: Running main() from gmock_main.cc 2024-09-11T08:34:19.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-09-11T08:34:19.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] Global test environment set-up. 2024-09-11T08:34:19.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-09-11T08:34:19.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-11T08:34:19.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.TooBig (11615 ms) 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (5441 ms) 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (5671 ms) 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3012 ms) 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (4223 ms) 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (4425 ms) 2024-09-11T08:34:19.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3511 ms) 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3192 ms) 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3530 ms) 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (2732 ms) 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3369 ms) 2024-09-11T08:34:19.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2621 ms) 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3021 ms) 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2985 ms) 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.Flush (2993 ms) 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3006 ms) 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-11T08:34:19.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2997 ms) 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3009 ms) 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3010 ms) 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3016 ms) 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3010 ms) 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-11T08:34:19.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3036 ms) 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3007 ms) 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3006 ms) 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2918 ms) 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 25 tests from LibRadosAio (92357 ms total) 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-11T08:34:19.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7360 ms) 2024-09-11T08:34:19.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[29355]: pgmap v202: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:19.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:19.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:19.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[29355]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:34:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:19.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[40592]: pgmap v202: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:19.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:19.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:19.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:19.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[40592]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:34:19.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:19 smithi172 ceph-mon[32987]: pgmap v202: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2413151561' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]: dispatch 2024-09-11T08:34:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-79089-26"}]': finished 2024-09-11T08:34:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:19 smithi172 ceph-mon[32987]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:34:19.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:20.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.884 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.885 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.885 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.885 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:20.885 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1585733850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:20.885 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:34:20.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-79089-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:20.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:34:20.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:20.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1585733850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:20.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1585733850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:20.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:22.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:22 smithi043 ceph-mon[40592]: pgmap v205: 268 pgs: 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:22.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:22 smithi043 ceph-mon[40592]: pgmap v206: 268 pgs: 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:34:22.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:22 smithi043 ceph-mon[40592]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T08:34:22.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:22 smithi043 ceph-mon[29355]: pgmap v205: 268 pgs: 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:22 smithi043 ceph-mon[29355]: pgmap v206: 268 pgs: 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:34:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:22 smithi043 ceph-mon[29355]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T08:34:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:22 smithi172 ceph-mon[32987]: pgmap v205: 268 pgs: 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:22 smithi172 ceph-mon[32987]: pgmap v206: 268 pgs: 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:34:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:22 smithi172 ceph-mon[32987]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T08:34:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-79112-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: pgmap v209: 300 pgs: 32 unknown, 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:34:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: pgmap v209: 300 pgs: 32 unknown, 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:23.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:34:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: pgmap v209: 300 pgs: 32 unknown, 8 creating+peering, 260 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:34:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-79335-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-79089-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:34:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:34:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:23] "GET /metrics HTTP/1.1" 200 235656 "" "Prometheus/2.43.0" 2024-09-11T08:34:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[40592]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:34:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2342886838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-79112-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[29355]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2342886838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-79112-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:24 smithi172 ceph-mon[32987]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:34:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:34:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]: dispatch 2024-09-11T08:34:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2342886838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-79112-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:34:25.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:34:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: pgmap v212: 316 pgs: 32 unknown, 16 creating+peering, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-11T08:34:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]': finished 2024-09-11T08:34:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2342886838' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-79112-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:34:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:25.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: pgmap v212: 316 pgs: 32 unknown, 16 creating+peering, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-11T08:34:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]': finished 2024-09-11T08:34:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2342886838' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-79112-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:34:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: pgmap v212: 316 pgs: 32 unknown, 16 creating+peering, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-13"}]': finished 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2342886838' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-79112-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:34:25.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:25.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:25.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:25.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:25.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:26.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForCompl (67790 ms total) 2024-09-11T08:34:26.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-11T08:34:26.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-11T08:34:26.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (502 ms) 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (502 ms total) 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (45 ms) 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (3 ms) 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (48 ms total) 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (29 ms) 2024-09-11T08:34:26.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (4 ms) 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (33 ms total) 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (42 ms) 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (8 ms) 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (50 ms total) 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-11T08:34:26.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1023 ms) 2024-09-11T08:34:26.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1023 ms total) 2024-09-11T08:34:26.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-11T08:34:26.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-11T08:34:26.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (106611 ms total) 2024-09-11T08:34:26.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-11T08:34:26.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:26.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:26.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:34:26.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[29355]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:34:26.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-11T08:34:26.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:26 smithi043 ceph-mon[40592]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:34:26.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:34:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2081268063' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2472769537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:26.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]: dispatch 2024-09-11T08:34:26.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:26.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]: dispatch 2024-09-11T08:34:26.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:26.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:26.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-79089-27"}]': finished 2024-09-11T08:34:26.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-79335-36"}]': finished 2024-09-11T08:34:26.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-11T08:34:26.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:26 smithi172 ceph-mon[32987]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:34:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: pgmap v215: 236 pgs: 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/845072000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: pgmap v215: 236 pgs: 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/845072000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:27.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:27.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:27.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:27.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:27.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:34:27.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: pgmap v215: 236 pgs: 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/845072000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-79112-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-79089-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:27.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:29.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:29.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:29.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:29.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:29.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:29.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:34:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: pgmap v218: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:34:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:30.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:34:30.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:30.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:30.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3241063920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:30.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:30.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: pgmap v218: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:30.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3241063920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: pgmap v218: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-79089-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-79173-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3241063920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:30.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: pgmap v221: 308 pgs: 40 unknown, 268 active+clean; 617 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 8.2 KiB/s wr, 11 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:34:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]': finished 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]: dispatch 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]: dispatch 2024-09-11T08:34:31.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:31.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: pgmap v221: 308 pgs: 40 unknown, 268 active+clean; 617 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 8.2 KiB/s wr, 11 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]': finished 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]: dispatch 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]: dispatch 2024-09-11T08:34:31.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:31.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: pgmap v221: 308 pgs: 40 unknown, 268 active+clean; 617 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 8.2 KiB/s wr, 11 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:34:31.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:31.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:34:31.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-15"}]': finished 2024-09-11T08:34:31.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-79112-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:31.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-11T08:34:31.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:34:31.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:31.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]: dispatch 2024-09-11T08:34:31.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:31.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]: dispatch 2024-09-11T08:34:31.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:32.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]': finished 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:32.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:32.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]': finished 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:32.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-15", "mode": "writeback"}]': finished 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1805019934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]: dispatch 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:32.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:32.512 INFO:tasks.workunit.client.0.smithi043.stdout:p: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-11T08:34:32.512 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-11T08:34:32.512 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4894 ms) 2024-09-11T08:34:32.512 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-11T08:34:32.512 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3123 ms) 2024-09-11T08:34:32.513 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-11T08:34:32.513 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3984 ms) 2024-09-11T08:34:32.513 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12001 ms total) 2024-09-11T08:34:32.513 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-11T08:34:32.513 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-11T08:34:32.513 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (112562 ms total) 2024-09-11T08:34:32.513 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-11T08:34:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: pgmap v224: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:34:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-11T08:34:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-11T08:34:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3242812361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: pgmap v224: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:34:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-11T08:34:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-11T08:34:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3242812361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:33.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:33.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: pgmap v224: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:34:33.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-11T08:34:33.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-11T08:34:33.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-79495-20"}]': finished 2024-09-11T08:34:33.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:33.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-79173-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-11T08:34:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:34:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/599095237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3242812361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]: dispatch 2024-09-11T08:34:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:33.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:34:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:33] "GET /metrics HTTP/1.1" 200 235747 "" "Prometheus/2.43.0" 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (11620 ms) 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (22761 ms) 2024-09-11T08:34:34.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (8565 ms) 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2683 ms) 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (2730 ms) 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3371 ms) 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2614 ms) 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2991 ms) 2024-09-11T08:34:34.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3013 ms) 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2990 ms) 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3020 ms) 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2986 ms) 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3006 ms) 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3010 ms) 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-11T08:34:34.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3022 ms) 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3471 ms) 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2572 ms) 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3008 ms) 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3006 ms) 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (2912 ms) 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3446 ms) 2024-09-11T08:34:34.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (2904 ms) 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2994 ms) 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3012 ms) 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3418 ms) 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2572 ms) 2024-09-11T08:34:34.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-11T08:34:34.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:34.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:34.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:34.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-79089-28"}]': finished 2024-09-11T08:34:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-79112-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2631353097' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]: dispatch 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:34:35.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: pgmap v227: 291 pgs: 32 creating+peering, 2 peering, 257 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]': finished 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:35.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-79173-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:35.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: pgmap v227: 291 pgs: 32 creating+peering, 2 peering, 257 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:35.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:35.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]': finished 2024-09-11T08:34:35.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:35.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:34:35.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:35.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:35.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:35.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:35.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-79173-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: pgmap v227: 291 pgs: 32 creating+peering, 2 peering, 257 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-15"}]': finished 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-79089-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:35.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-79173-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-79173-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:36 smithi043 ceph-mon[40592]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:34:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-79173-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3106376500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-79112-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-79173-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:36 smithi043 ceph-mon[29355]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:34:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-79173-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3106376500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-79112-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-79173-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:36 smithi172 ceph-mon[32987]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:34:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-79173-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3106376500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-79112-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: pgmap v230: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3106376500' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-79112-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-79173-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[29355]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: pgmap v230: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:37.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3106376500' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-79112-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:34:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-79173-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:37 smithi043 ceph-mon[40592]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:34:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: pgmap v230: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-79089-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3106376500' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-79112-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:34:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-79173-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:37 smithi172 ceph-mon[32987]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:34:38.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAiexpected=6 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:bd63b0f1:::8:head 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:bd63b0f1:::8:head expected=12:bd63b0f1:::8:head 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:bd63b0f1:::8:head -> 8 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=8 expected=8 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:b29083e3:::5:head 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:b29083e3:::5:head expected=12:b29083e3:::5:head 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:b29083e3:::5:head -> 5 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=5 expected=5 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:89d3ae78:::11:head 2024-09-11T08:34:38.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:89d3ae78:::11:head expected=12:89d3ae78:::11:head 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:89d3ae78:::11:head -> 11 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=11 expected=11 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:863748b0:::15:head 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:863748b0:::15:head expected=12:863748b0:::15:head 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:863748b0:::15:head -> 15 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=15 expected=15 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:6cac518f:::0:head 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:6cac518f:::0:head expected=12:6cac518f:::0:head 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:6cac518f:::0:head -> 0 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=0 expected=0 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:5c6b0b28:::7:head 2024-09-11T08:34:38.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:5c6b0b28:::7:head expected=12:5c6b0b28:::7:head 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:5c6b0b28:::7:head -> 7 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=7 expected=7 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:62a1935d:::14:head 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:62a1935d:::14:head expected=12:62a1935d:::14:head 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:62a1935d:::14:head -> 14 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=14 expected=14 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 12:52ea6a34:::10:head 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=12:52ea6a34:::10:head expected=12:52ea6a34:::10:head 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 12:52ea6a34:::10:head -> 10 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=10 expected=10 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (230 ms) 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-11T08:34:38.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (95397 ms) 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 0/5 -> MIN 12:33333333::::head 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 1/5 -> 12:33333333::::head 12:66666666::::head 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 2/5 -> 12:66666666::::head 12:99999999::::head 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 3/5 -> 12:99999999::::head 12:cccccccc::::head 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 4/5 -> 12:cccccccc::::head MAX 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (6015 ms) 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 7 tests from LibRadosList (102149 ms total) 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1022 ms) 2024-09-11T08:34:38.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo2,foo3 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo2 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo3 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo4,foo5 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo4 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo5 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo6,foo7 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo7 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo6 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo4 2024-09-11T08:34:38.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo5 2024-09-11T08:34:38.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo7 2024-09-11T08:34:38.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo6 2024-09-11T08:34:38.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo1 2024-09-11T08:34:38.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo1 2024-09-11T08:34:38.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo2 2024-09-11T08:34:38.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo3 2024-09-11T08:34:38.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (32 ms) 2024-09-11T08:34:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:34:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/749259948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-79112-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/749259948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-79112-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:38.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:38.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:34:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/749259948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-79112-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:38.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:38.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:38.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:38.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:39.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: pgmap v233: 244 pgs: 48 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:39.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/749259948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-79112-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]': finished 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]: dispatch 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]: dispatch 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: pgmap v233: 244 pgs: 48 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:39.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/749259948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-79112-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:39.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:39.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]': finished 2024-09-11T08:34:39.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:34:39.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:39.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:39.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]: dispatch 2024-09-11T08:34:39.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:39.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]: dispatch 2024-09-11T08:34:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: pgmap v233: 244 pgs: 48 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/749259948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-79112-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-17"}]': finished 2024-09-11T08:34:39.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:34:39.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3022787806' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:39.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:39.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]: dispatch 2024-09-11T08:34:39.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]: dispatch 2024-09-11T08:34:39.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]: dispatch 2024-09-11T08:34:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:40 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]': finished 2024-09-11T08:34:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:40 smithi172 ceph-mon[32987]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:34:41.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:41.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:41.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:41.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:41.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:41.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]': finished 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[29355]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-79089-29"}]': finished 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-17", "mode": "writeback"}]': finished 2024-09-11T08:34:41.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[40592]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:34:41.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]: dispatch 2024-09-11T08:34:41.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:41.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: pgmap v236: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:42.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/982659035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-79112-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:42.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:42.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: pgmap v236: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:42.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:34:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/982659035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-79112-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: pgmap v236: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:42.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3667831918' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-79173-2"}]': finished 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-79089-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/982659035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-79112-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]: dispatch 2024-09-11T08:34:42.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/982659035' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-79112-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]': finished 2024-09-11T08:34:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:34:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3706489711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/982659035' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-79112-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]': finished 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3706489711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/982659035' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-79112-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-17"}]': finished 2024-09-11T08:34:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:34:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3706489711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:34:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:43] "GET /metrics HTTP/1.1" 200 230864 "" "Prometheus/2.43.0" 2024-09-11T08:34:44.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-09-11T08:34:44.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 1 0 2024-09-11T08:34:44.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 10 0 2024-09-11T08:34:44.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 13 0 2024-09-11T08:34:44.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 7 0 2024-09-11T08:34:44.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 14 0 2024-09-11T08:34:44.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 0 0 2024-09-11T08:34:44.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 15 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 11 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 5 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 8 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 6 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 3 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 4 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 12 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 9 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2 0 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (42 ms) 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 3 tests from LibRadosListEC (1096 ms total) 2024-09-11T08:34:44.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-09-11T08:34:44.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-11T08:34:44.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-11T08:34:44.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3007 ms) 2024-09-11T08:34:44.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 1 test from LibRadosListNP (3007 ms total) 2024-09-11T08:34:44.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-09-11T08:34:44.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] Global test environment tear-down 2024-09-11T08:34:44.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [==========] 11 tests from 3 test suites ran. (124665 ms total) 2024-09-11T08:34:44.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ PASSED ] 11 tests. 2024-09-11T08:34:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[29355]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:34:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3706489711' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:34:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:34:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[40592]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:34:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3706489711' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:34:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:34:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-79089-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79173-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:44 smithi172 ceph-mon[32987]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:34:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3706489711' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:34:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:34:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:34:45.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:34:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-mon[32987]: pgmap v242: 236 pgs: 32 creating+peering, 8 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-11T08:34:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-mon[32987]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:34:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3221616798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[29355]: pgmap v242: 236 pgs: 32 creating+peering, 8 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-11T08:34:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[29355]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:34:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3221616798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[40592]: pgmap v242: 236 pgs: 32 creating+peering, 8 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-79173-3","pool2":"test-rados-api-smithi043-79173-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[40592]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3221616798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:34:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3221616798' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]': finished 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:46.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:46.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:34:46.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:46.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:46.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3221616798' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:34:46.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]': finished 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:46.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-79112-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:34:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3221616798' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi043-79112-33", "var": "eio", "val": "true"}]': finished 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2161045112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]: dispatch 2024-09-11T08:34:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:47 smithi043 ceph-mon[40592]: pgmap v245: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:47.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:47.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:47 smithi043 ceph-mon[29355]: pgmap v245: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:47 smithi172 ceph-mon[32987]: pgmap v245: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]': finished 2024-09-11T08:34:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:34:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]: dispatch 2024-09-11T08:34:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]: dispatch 2024-09-11T08:34:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]': finished 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]: dispatch 2024-09-11T08:34:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]: dispatch 2024-09-11T08:34:48.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:48.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-19"}]': finished 2024-09-11T08:34:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-79089-30"}]': finished 2024-09-11T08:34:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:34:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]: dispatch 2024-09-11T08:34:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]: dispatch 2024-09-11T08:34:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: pgmap v248: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]': finished 2024-09-11T08:34:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3353072264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-79112-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: pgmap v248: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]': finished 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:34:50.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3353072264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-79112-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:50.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:50.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:50.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:50.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: pgmap v248: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-19", "mode": "writeback"}]': finished 2024-09-11T08:34:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-79089-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:34:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3353072264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-79112-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:50.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3353072264' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-79112-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:34:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: pgmap v251: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]': finished 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[40592]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3353072264' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-79112-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:51.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: pgmap v251: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:51.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:51.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:51.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]': finished 2024-09-11T08:34:51.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:51 smithi043 ceph-mon[29355]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:34:51.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3353072264' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-79112-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:34:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:51.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]: dispatch 2024-09-11T08:34:51.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: pgmap v251: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:51.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:51.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-79089-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:51.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-19"}]': finished 2024-09-11T08:34:51.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:51 smithi172 ceph-mon[32987]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:34:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:52 smithi043 ceph-mon[40592]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:34:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1153800385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:52 smithi043 ceph-mon[29355]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:34:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1153800385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:52.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:52 smithi172 ceph-mon[32987]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:34:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1153800385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:53.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:53] "GET /metrics HTTP/1.1" 200 230864 "" "Prometheus/2.43.0" 2024-09-11T08:34:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[40592]: pgmap v254: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[40592]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:34:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[29355]: pgmap v254: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[29355]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:53 smithi172 ceph-mon[32987]: pgmap v254: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:34:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:53 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:34:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-79112-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:53 smithi172 ceph-mon[32987]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:34:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:53.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[40592]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[29355]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:34:54.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:54 smithi172 ceph-mon[32987]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:34:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/130890140' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]: dispatch 2024-09-11T08:34:54.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:54.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:34:54.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-79089-31"}]': finished 2024-09-11T08:34:55.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:34:55.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:34:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:34:55.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: pgmap v257: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:55.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2044239241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:55.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:55.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]': finished 2024-09-11T08:34:55.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:55.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]: dispatch 2024-09-11T08:34:55.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:55.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:34:55.734 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]: dispatch 2024-09-11T08:34:55.735 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: pgmap v257: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2044239241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]': finished 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]: dispatch 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:34:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: pgmap v257: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2044239241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-21"}]': finished 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]: dispatch 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-79089-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:34:55.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]: dispatch 2024-09-11T08:34:55.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:55.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]: dispatch 2024-09-11T08:34:55.899 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:34:57.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:57.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:57 smithi043 ceph-mon[40592]: pgmap v260: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:57.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:57 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:57.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]': finished 2024-09-11T08:34:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:57 smithi043 ceph-mon[29355]: pgmap v260: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:57 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]': finished 2024-09-11T08:34:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:34:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:57 smithi172 ceph-mon[32987]: pgmap v260: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:34:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:57 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:34:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-21", "mode": "writeback"}]': finished 2024-09-11T08:34:58.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:58.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:34:58.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:58.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3073896537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3073896537' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:34:58.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:58.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3073896537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:58.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:58.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:34:58.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:58.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3073896537' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:58.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:34:58.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:58.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3073896537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-79089-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3073896537' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:58.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]: dispatch 2024-09-11T08:34:59.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[40592]: pgmap v263: 300 pgs: 72 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:59.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:59.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]': finished 2024-09-11T08:34:59.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[40592]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:34:59.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/344319583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:59.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[29355]: pgmap v263: 300 pgs: 72 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]': finished 2024-09-11T08:34:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[29355]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:34:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/344319583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:34:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:59 smithi172 ceph-mon[32987]: pgmap v263: 300 pgs: 72 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:34:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:59 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:34:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-21"}]': finished 2024-09-11T08:34:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:59 smithi172 ceph-mon[32987]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:34:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/344319583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:34:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:34:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:00.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:00.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[40592]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:35:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[29355]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:35:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:00 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-79112-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:00 smithi172 ceph-mon[32987]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:35:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:01.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[40592]: pgmap v266: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:01.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:35:01.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[40592]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:35:01.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:01.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[29355]: pgmap v266: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[29355]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:01.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:01 smithi172 ceph-mon[32987]: pgmap v266: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:35:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:01 smithi172 ceph-mon[32987]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:35:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2927402402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:01.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]: dispatch 2024-09-11T08:35:01.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:35:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:35:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[29355]: pgmap v269: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:02.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:02.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:02 smithi043 ceph-mon[40592]: pgmap v269: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-79089-32"}]': finished 2024-09-11T08:35:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:35:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:02 smithi172 ceph-mon[32987]: pgmap v269: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:04.029 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:35:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:03] "GET /metrics HTTP/1.1" 200 233342 "" "Prometheus/2.43.0" 2024-09-11T08:35:04.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:04.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[40592]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[29355]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:04.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:04.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-79089-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:04.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:04 smithi172 ceph-mon[32987]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:35:04.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:04.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:04.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:04.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:05.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:05.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/691395934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: pgmap v272: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/691395934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]': finished 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]: dispatch 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]: dispatch 2024-09-11T08:35:05.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/691395934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: pgmap v272: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/691395934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]': finished 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]: dispatch 2024-09-11T08:35:05.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]: dispatch 2024-09-11T08:35:05.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:05.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:35:05.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:05.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/691395934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:05.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:05.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: pgmap v272: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:35:05.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-79089-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/691395934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-23"}]': finished 2024-09-11T08:35:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:35:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]: dispatch 2024-09-11T08:35:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]: dispatch 2024-09-11T08:35:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:35:05.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:35:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:35:06.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:06.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]': finished 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[40592]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1976180753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]': finished 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[29355]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1976180753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:06.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:06 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:06 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-23", "mode": "writeback"}]': finished 2024-09-11T08:35:06.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:06 smithi172 ceph-mon[32987]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:35:06.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1976180753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:06.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:07.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:07.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[29355]: pgmap v275: 300 pgs: 72 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[29355]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[40592]: pgmap v275: 300 pgs: 72 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:07.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[40592]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:35:07.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:07.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:07.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:07.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:07 smithi172 ceph-mon[32987]: pgmap v275: 300 pgs: 72 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:35:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:07.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:07 smithi172 ceph-mon[32987]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:35:07.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:07.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:07.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:07.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:35:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/389550511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[40592]: pgmap v278: 324 pgs: 96 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:08.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/389550511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:08.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:08 smithi043 ceph-mon[29355]: pgmap v278: 324 pgs: 96 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:35:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/88720710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]: dispatch 2024-09-11T08:35:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]: dispatch 2024-09-11T08:35:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/389550511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:08 smithi172 ceph-mon[32987]: pgmap v278: 324 pgs: 96 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]': finished 2024-09-11T08:35:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/389550511' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]': finished 2024-09-11T08:35:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/389550511' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:35:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-79089-33"}]': finished 2024-09-11T08:35:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-23"}]': finished 2024-09-11T08:35:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/389550511' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:35:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:11.398 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:11.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[40592]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:35:11.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:11.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:11.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1369802542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:11.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:11.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[40592]: pgmap v281: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:35:11.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:11.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[29355]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:35:11.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:11.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:11.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1369802542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:11.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:11.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:11 smithi043 ceph-mon[29355]: pgmap v281: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:35:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-79089-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:11 smithi172 ceph-mon[32987]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:35:11.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:11.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:11.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1369802542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:11.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:11.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:11 smithi172 ceph-mon[32987]: pgmap v281: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:35:12.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:12.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[40592]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[40592]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1726017292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[29355]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:12.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:12.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:12.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:12.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[29355]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:35:12.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1726017292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:12 smithi172 ceph-mon[32987]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:35:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:12.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:12.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-79089-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:12.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:12.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:12 smithi172 ceph-mon[32987]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:35:12.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1726017292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:13 smithi172 ceph-mon[32987]: pgmap v284: 396 pgs: 104 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:35:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:13 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:13 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1726017292' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:13 smithi172 ceph-mon[32987]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:35:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:13.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:13 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[40592]: pgmap v284: 396 pgs: 104 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:35:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1726017292' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[40592]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[29355]: pgmap v284: 396 pgs: 104 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1726017292' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-79112-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[29355]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:35:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:35:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:13] "GET /metrics HTTP/1.1" 200 243036 "" "Prometheus/2.43.0" 2024-09-11T08:35:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[40592]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:35:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:14.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:14.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[29355]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:35:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:14 smithi172 ceph-mon[32987]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:35:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:14.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:35:15.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:35:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-mon[32987]: pgmap v287: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]': finished 2024-09-11T08:35:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-mon[32987]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T08:35:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]: dispatch 2024-09-11T08:35:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]: dispatch 2024-09-11T08:35:15.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[40592]: pgmap v287: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]': finished 2024-09-11T08:35:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[40592]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]: dispatch 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]: dispatch 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[29355]: pgmap v287: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-25"}]': finished 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[29355]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T08:35:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]: dispatch 2024-09-11T08:35:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2487420339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]: dispatch 2024-09-11T08:35:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]: dispatch 2024-09-11T08:35:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]': finished 2024-09-11T08:35:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]': finished 2024-09-11T08:35:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-25", "mode": "writeback"}]': finished 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-79089-34"}]': finished 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: pgmap v290: 292 pgs: 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T08:35:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: pgmap v290: 292 pgs: 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T08:35:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: pgmap v290: 292 pgs: 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-79089-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]: dispatch 2024-09-11T08:35:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:18.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:18 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]': finished 2024-09-11T08:35:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:18 smithi172 ceph-mon[32987]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T08:35:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:18 smithi172 ceph-mon[32987]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T08:35:19.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2185546820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-79112-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:19.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:19.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]': finished 2024-09-11T08:35:19.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[40592]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T08:35:19.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:19.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[40592]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T08:35:19.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2185546820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-79112-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-25"}]': finished 2024-09-11T08:35:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[29355]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T08:35:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-79089-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[29355]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T08:35:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2185546820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-79112-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:19 smithi172 ceph-mon[32987]: pgmap v293: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2185546820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-79112-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:19 smithi172 ceph-mon[32987]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T08:35:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:20.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[29355]: pgmap v293: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:20.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2185546820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-79112-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:20.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[29355]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T08:35:20.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:20.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:20.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[40592]: pgmap v293: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:20.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2185546820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-79112-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:20.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[40592]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T08:35:20.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:20.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:21.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:21.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:21.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[40592]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T08:35:21.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:21.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[29355]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:21.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:21.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:21 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:21 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:21 smithi172 ceph-mon[32987]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T08:35:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:21.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:21 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:21.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:21 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:21.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:21 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:22.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: pgmap v296: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:22.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:22.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3986655301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: pgmap v296: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:22.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T08:35:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3986655301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:22.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: pgmap v296: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T08:35:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2574821880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]: dispatch 2024-09-11T08:35:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3986655301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:22.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:23.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:23] "GET /metrics HTTP/1.1" 200 243036 "" "Prometheus/2.43.0" 2024-09-11T08:35:23.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: pgmap v299: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: pgmap v299: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:23.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: pgmap v299: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-79089-35"}]': finished 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-79112-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:24.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]': finished 2024-09-11T08:35:24.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:24.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T08:35:24.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]: dispatch 2024-09-11T08:35:24.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]: dispatch 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]': finished 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]: dispatch 2024-09-11T08:35:24.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]: dispatch 2024-09-11T08:35:24.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:24.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:24.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:24.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:24.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-27"}]': finished 2024-09-11T08:35:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:24.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T08:35:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]: dispatch 2024-09-11T08:35:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]: dispatch 2024-09-11T08:35:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:24.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:25.665 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:35:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:35:25.666 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:25 smithi172 ceph-mon[32987]: pgmap v302: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:35:25.666 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:25 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:25.666 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]': finished 2024-09-11T08:35:25.666 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:25.666 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:25 smithi172 ceph-mon[32987]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T08:35:25.666 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[40592]: pgmap v302: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:35:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]': finished 2024-09-11T08:35:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[40592]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T08:35:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[29355]: pgmap v302: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:35:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-27", "mode": "writeback"}]': finished 2024-09-11T08:35:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[29355]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T08:35:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:25.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:35:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:35:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:26 smithi043 ceph-mon[40592]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T08:35:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:26 smithi043 ceph-mon[29355]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T08:35:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:26.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-79089-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:26.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:26.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:26 smithi172 ceph-mon[32987]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T08:35:26.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: pgmap v305: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: pgmap v305: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:35:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T08:35:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: pgmap v305: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:35:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T08:35:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]: dispatch 2024-09-11T08:35:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]': finished 2024-09-11T08:35:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[40592]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T08:35:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]': finished 2024-09-11T08:35:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[29355]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T08:35:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:28 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-27"}]': finished 2024-09-11T08:35:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:28 smithi172 ceph-mon[32987]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T08:35:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:28.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[40592]: pgmap v308: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[40592]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T08:35:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[29355]: pgmap v308: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[29355]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T08:35:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:29 smithi172 ceph-mon[32987]: pgmap v308: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:29 smithi172 ceph-mon[32987]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T08:35:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1074119993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]: dispatch 2024-09-11T08:35:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[40592]: pgmap v311: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:30.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:30.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:30 smithi043 ceph-mon[29355]: pgmap v311: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-79089-36"}]': finished 2024-09-11T08:35:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:30 smithi172 ceph-mon[32987]: pgmap v311: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T08:35:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:31.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:31.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:31.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:31.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:31.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T08:35:31.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:31.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:31.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:31.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:31.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3421242796' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-46"}]': finished 2024-09-11T08:35:31.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:31.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-79089-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:31.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T08:35:31.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:31.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:31.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:31.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:31.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:32 smithi172 ceph-mon[32987]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T08:35:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:32 smithi043 ceph-mon[40592]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T08:35:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:33.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:33.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-79112-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:33.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:32 smithi043 ceph-mon[29355]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T08:35:33.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:33.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-79089-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:33] "GET /metrics HTTP/1.1" 200 233328 "" "Prometheus/2.43.0" 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: pgmap v314: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:34.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: pgmap v314: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:34.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T08:35:34.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:34.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:34.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:34.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:34.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:34.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T08:35:34.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:34.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: pgmap v314: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T08:35:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:34.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:34.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:34.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-79112-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:34.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:34.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T08:35:34.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:34.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:35.676 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:35:35.677 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-mon[32987]: pgmap v317: 244 pgs: 8 unknown, 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-11T08:35:35.677 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]': finished 2024-09-11T08:35:35.677 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-mon[32987]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T08:35:35.677 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]: dispatch 2024-09-11T08:35:35.677 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:35.677 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]: dispatch 2024-09-11T08:35:35.677 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[40592]: pgmap v317: 244 pgs: 8 unknown, 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-11T08:35:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]': finished 2024-09-11T08:35:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[40592]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T08:35:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]: dispatch 2024-09-11T08:35:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]: dispatch 2024-09-11T08:35:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:35.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[29355]: pgmap v317: 244 pgs: 8 unknown, 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-11T08:35:35.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-29"}]': finished 2024-09-11T08:35:35.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[29355]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T08:35:35.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]: dispatch 2024-09-11T08:35:35.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:35.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]: dispatch 2024-09-11T08:35:35.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:35.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:35:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:35:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]': finished 2024-09-11T08:35:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[40592]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T08:35:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:36.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]': finished 2024-09-11T08:35:36.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:36.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[29355]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T08:35:36.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:36.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:36.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:36 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-29", "mode": "writeback"}]': finished 2024-09-11T08:35:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:36 smithi172 ceph-mon[32987]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T08:35:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1802908483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:36.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]: dispatch 2024-09-11T08:35:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: pgmap v320: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: pgmap v320: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: pgmap v320: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:35:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-79089-37"}]': finished 2024-09-11T08:35:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T08:35:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]: dispatch 2024-09-11T08:35:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:39.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:39.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:39.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T08:35:39.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:39.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:39.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:39.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:39.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:39.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:39.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T08:35:39.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/391873733' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-79112-47"}]': finished 2024-09-11T08:35:39.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-79089-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:39.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:40.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: pgmap v323: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T08:35:40.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:40.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:40.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: pgmap v323: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:40.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T08:35:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: pgmap v323: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-79112-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-79089-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T08:35:40.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:40.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]: dispatch 2024-09-11T08:35:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[40592]: pgmap v326: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:35:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]': finished 2024-09-11T08:35:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[40592]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T08:35:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[29355]: pgmap v326: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:35:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]': finished 2024-09-11T08:35:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:40 smithi043 ceph-mon[29355]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T08:35:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:40 smithi172 ceph-mon[32987]: pgmap v326: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:35:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:40 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-79112-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-29"}]': finished 2024-09-11T08:35:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:40 smithi172 ceph-mon[32987]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T08:35:42.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:41 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:42.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:41 smithi043 ceph-mon[40592]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T08:35:42.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:42.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:41 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:41 smithi043 ceph-mon[29355]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T08:35:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:42.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:41 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:41 smithi172 ceph-mon[32987]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T08:35:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: pgmap v329: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T08:35:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: pgmap v329: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: pgmap v329: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/380757265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:43.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]: dispatch 2024-09-11T08:35:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:35:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:43] "GET /metrics HTTP/1.1" 200 228520 "" "Prometheus/2.43.0" 2024-09-11T08:35:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:44.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:44.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:44.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T08:35:44.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:44.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:44.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:44.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:44.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:44.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-79089-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:44.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:44.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:44.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:44.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:44.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T08:35:44.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:44.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:44.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:44.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:44.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:44.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-79089-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-79089-38"}]': finished 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1141257570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]: dispatch 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-79089-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:45.687 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:35:45.959 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:35:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: pgmap v332: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-79089-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T08:35:46.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:46.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-79089-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:46.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:46.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:46.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:46.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-79112-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:46.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: pgmap v332: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:46.549 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:46.549 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:46.549 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-79089-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:46.549 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T08:35:46.549 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:46.549 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-79089-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:46.549 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-79112-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: pgmap v332: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-79112-48"}]': finished 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-79089-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:46.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-79089-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:46.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:46.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:46.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:46.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-79112-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]': finished 2024-09-11T08:35:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-79112-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:46 smithi172 ceph-mon[32987]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T08:35:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]: dispatch 2024-09-11T08:35:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-79112-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]: dispatch 2024-09-11T08:35:47.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:46 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:47.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-79089-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:47.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]': finished 2024-09-11T08:35:47.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-79112-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:47.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[40592]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T08:35:47.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]: dispatch 2024-09-11T08:35:47.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-79112-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]: dispatch 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-79089-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-31"}]': finished 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-79112-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[29355]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]: dispatch 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-79112-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]: dispatch 2024-09-11T08:35:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-79089-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: pgmap v335: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:35:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]': finished 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-79112-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:48.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:48.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: pgmap v335: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]': finished 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-79112-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: pgmap v335: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-31", "mode": "writeback"}]': finished 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-79112-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]: dispatch 2024-09-11T08:35:49.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:48 smithi172 ceph-mon[32987]: pgmap v338: 244 pgs: 16 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:49.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:48 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:49.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]': finished 2024-09-11T08:35:49.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:48 smithi172 ceph-mon[32987]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T08:35:49.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:49.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[29355]: pgmap v338: 244 pgs: 16 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:49.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:49.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]': finished 2024-09-11T08:35:49.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[29355]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T08:35:49.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:49.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[40592]: pgmap v338: 244 pgs: 16 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:35:49.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:49.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-31"}]': finished 2024-09-11T08:35:49.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[40592]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T08:35:49.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:50.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:50.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:50 smithi043 ceph-mon[40592]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T08:35:50.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:50.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:50.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:50.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:50 smithi043 ceph-mon[29355]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T08:35:50.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:50.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:50 smithi172 ceph-mon[32987]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T08:35:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]: dispatch 2024-09-11T08:35:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: pgmap v341: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:51.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-79089-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: pgmap v341: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:51.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:51.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-79089-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: pgmap v341: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:35:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3477106652' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-79089-39"}]': finished 2024-09-11T08:35:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T08:35:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]: dispatch 2024-09-11T08:35:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-79089-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:53.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:53.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:53.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-79089-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:53.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T08:35:53.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:53.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-79089-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:53.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:53.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:53.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:53.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-79112-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:53.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:53.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:53.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-79089-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:53.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T08:35:53.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-79089-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-79112-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3942930181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-79112-49"}]': finished 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-79089-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-79089-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:35:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-79112-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:53] "GET /metrics HTTP/1.1" 200 228520 "" "Prometheus/2.43.0" 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: pgmap v344: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-79112-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-79112-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-79089-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]': finished 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]: dispatch 2024-09-11T08:35:54.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]: dispatch 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: pgmap v344: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-79112-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-79112-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-79089-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]': finished 2024-09-11T08:35:54.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T08:35:54.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]: dispatch 2024-09-11T08:35:54.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]: dispatch 2024-09-11T08:35:54.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: pgmap v344: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:35:54.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-79112-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-79112-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-79089-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-33"}]': finished 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]: dispatch 2024-09-11T08:35:54.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]: dispatch 2024-09-11T08:35:55.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:54 smithi172 ceph-mon[32987]: pgmap v347: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:55.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:54 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:55.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-79112-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:55.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]': finished 2024-09-11T08:35:55.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:54 smithi172 ceph-mon[32987]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T08:35:55.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:55.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:55.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[40592]: pgmap v347: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:55.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:55.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-79112-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]': finished 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[40592]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[29355]: pgmap v347: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-79112-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-33", "mode": "writeback"}]': finished 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[29355]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T08:35:55.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:55.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:35:55.698 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:35:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:35:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:35:55.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:35:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:35:56.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:55 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:56.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:56.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:55 smithi172 ceph-mon[32987]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T08:35:56.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:56.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:56.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:56.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:56.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:56.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[40592]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T08:35:56.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:56.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:56.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:56.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:35:56.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:35:56.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[29355]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T08:35:56.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:56.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:56.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]: dispatch 2024-09-11T08:35:57.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:57.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[40592]: pgmap v350: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:57.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:57.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]': finished 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[40592]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[29355]: pgmap v350: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]': finished 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[29355]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T08:35:57.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:57.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:56 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:35:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:56 smithi172 ceph-mon[32987]: pgmap v350: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:35:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:56 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:35:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-33"}]': finished 2024-09-11T08:35:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:56 smithi172 ceph-mon[32987]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T08:35:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]: dispatch 2024-09-11T08:35:57.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:58.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:58.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:58.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:58.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[40592]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T08:35:58.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[29355]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T08:35:58.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:35:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2380095950' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-79089-40"}]': finished 2024-09-11T08:35:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:57 smithi172 ceph-mon[32987]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T08:35:58.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]: dispatch 2024-09-11T08:35:59.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-79089-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-79089-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-79089-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-79089-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:59.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-79089-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T08:35:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-79089-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-79089-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:35:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3926741185' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-79112-50"}]': finished 2024-09-11T08:35:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-79089-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:35:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T08:35:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-79089-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:35:59.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[29355]: pgmap v353: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[29355]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T08:36:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:00.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3167208059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[40592]: pgmap v353: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[40592]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T08:36:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3167208059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:00.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:35:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:59 smithi172 ceph-mon[32987]: pgmap v353: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:59 smithi172 ceph-mon[32987]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T08:36:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3167208059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:35:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[29355]: pgmap v356: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-79089-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3167208059' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[29355]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[40592]: pgmap v356: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-79089-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3167208059' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[40592]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T08:36:01.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:01.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:01.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:01 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:01 smithi172 ceph-mon[32987]: pgmap v356: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-79089-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3167208059' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-79112-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:01 smithi172 ceph-mon[32987]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T08:36:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:01.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:01 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[40592]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T08:36:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]: dispatch 2024-09-11T08:36:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]: dispatch 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[29355]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]: dispatch 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]: dispatch 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:02.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:02 smithi172 ceph-mon[32987]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T08:36:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]: dispatch 2024-09-11T08:36:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]: dispatch 2024-09-11T08:36:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:02.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:03.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:03] "GET /metrics HTTP/1.1" 200 228507 "" "Prometheus/2.43.0" 2024-09-11T08:36:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[40592]: pgmap v359: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]': finished 2024-09-11T08:36:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[40592]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-79112-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[29355]: pgmap v359: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]': finished 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[29355]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-79112-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:36:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:03 smithi172 ceph-mon[32987]: pgmap v359: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:03 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-35", "mode": "writeback"}]': finished 2024-09-11T08:36:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:03.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:03 smithi172 ceph-mon[32987]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T08:36:03.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-79112-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:03.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:36:04.581 INFO:tasks.workunit.client.0.smithi043.stdout:ete 2024-09-11T08:36:04.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6998 ms) 2024-09-11T08:36:04.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-11T08:36:04.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6994 ms) 2024-09-11T08:36:04.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-11T08:36:04.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6997 ms) 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7036 ms) 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7027 ms) 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7016 ms) 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7014 ms) 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6923 ms) 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7013 ms) 2024-09-11T08:36:04.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7047 ms) 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6943 ms) 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6989 ms) 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7011 ms) 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7449 ms) 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6602 ms) 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112420 ms total) 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: 2024-09-11T08:36:04.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] Global test environment tear-down 2024-09-11T08:36:04.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (204777 ms total) 2024-09-11T08:36:04.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ PASSED ] 41 tests. 2024-09-11T08:36:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:04 smithi043 ceph-mon[29355]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T08:36:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:36:04.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:04 smithi043 ceph-mon[40592]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T08:36:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:36:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:04 smithi172 ceph-mon[32987]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T08:36:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]: dispatch 2024-09-11T08:36:05.709 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:36:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:36:05.709 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:05 smithi172 ceph-mon[32987]: pgmap v362: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:05.709 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-79112-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:05.709 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:05.709 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:05 smithi172 ceph-mon[32987]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T08:36:05.709 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:05.710 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:05.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[40592]: pgmap v362: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-79112-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[40592]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T08:36:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:05.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[29355]: pgmap v362: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-79112-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2454561621' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-79089-41"}]': finished 2024-09-11T08:36:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[29355]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T08:36:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:05.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:05.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:36:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:36:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[40592]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T08:36:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[40592]: pgmap v365: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[29355]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T08:36:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:06 smithi043 ceph-mon[29355]: pgmap v365: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:06.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:06.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:06 smithi172 ceph-mon[32987]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T08:36:06.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:06.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:06 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:06.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:06 smithi172 ceph-mon[32987]: pgmap v365: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:36:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:07 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:07 smithi043 ceph-mon[40592]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T08:36:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:07 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:07 smithi043 ceph-mon[29355]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T08:36:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:07 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:07 smithi172 ceph-mon[32987]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T08:36:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[40592]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[40592]: pgmap v368: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[29355]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:08 smithi043 ceph-mon[29355]: pgmap v368: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:08 smithi172 ceph-mon[32987]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T08:36:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]: dispatch 2024-09-11T08:36:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:08.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:08 smithi172 ceph-mon[32987]: pgmap v368: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[40592]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[29355]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T08:36:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:09.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2675813109' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-79112-52"}]': finished 2024-09-11T08:36:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:09 smithi172 ceph-mon[32987]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T08:36:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]: dispatch 2024-09-11T08:36:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:11 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:11 smithi172 ceph-mon[32987]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T08:36:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-79112-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:11.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:11 smithi172 ceph-mon[32987]: pgmap v371: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:36:11.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:11.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:11.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:11.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[40592]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T08:36:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-79112-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[40592]: pgmap v371: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:36:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-35"}]': finished 2024-09-11T08:36:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[29355]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T08:36:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-79112-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:11 smithi043 ceph-mon[29355]: pgmap v371: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:36:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:12 smithi172 ceph-mon[32987]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T08:36:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-79112-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:12 smithi172 ceph-mon[32987]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T08:36:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:12.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[40592]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T08:36:12.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-79112-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:12.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[40592]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T08:36:12.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:12.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:12.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:12.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[29355]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T08:36:12.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-79112-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:12.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[29355]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T08:36:12.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:12.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:12.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:13 smithi172 ceph-mon[32987]: pgmap v374: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:36:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:13 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:13 smithi172 ceph-mon[32987]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T08:36:13.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:13.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[40592]: pgmap v374: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:36:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[40592]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T08:36:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[29355]: pgmap v374: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:36:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[29355]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T08:36:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:36:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:13] "GET /metrics HTTP/1.1" 200 230939 "" "Prometheus/2.43.0" 2024-09-11T08:36:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[40592]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T08:36:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[29355]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T08:36:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:14 smithi172 ceph-mon[32987]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T08:36:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:15.720 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:36:15.720 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-mon[32987]: pgmap v377: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:36:15.721 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]': finished 2024-09-11T08:36:15.721 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:15.721 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-mon[32987]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T08:36:15.721 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:15.721 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]: dispatch 2024-09-11T08:36:15.721 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]: dispatch 2024-09-11T08:36:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[40592]: pgmap v377: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:36:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]': finished 2024-09-11T08:36:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[40592]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T08:36:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]: dispatch 2024-09-11T08:36:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]: dispatch 2024-09-11T08:36:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[29355]: pgmap v377: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:36:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-37"}]': finished 2024-09-11T08:36:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[29355]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T08:36:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]: dispatch 2024-09-11T08:36:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]: dispatch 2024-09-11T08:36:15.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]: dispatch 2024-09-11T08:36:15.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:36:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:36:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]': finished 2024-09-11T08:36:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T08:36:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-79112-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]': finished 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-79112-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:16.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1715673244' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-79112-53"}]': finished 2024-09-11T08:36:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-37", "mode": "writeback"}]': finished 2024-09-11T08:36:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T08:36:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-79112-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:16.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:17 smithi172 ceph-mon[32987]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-79112-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:17 smithi172 ceph-mon[32987]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T08:36:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-79112-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:18.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[40592]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-79112-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:18.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[40592]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-79112-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[29355]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-79112-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[29355]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T08:36:18.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:18.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-79112-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:18.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]: dispatch 2024-09-11T08:36:19.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:19.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]': finished 2024-09-11T08:36:19.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[40592]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T08:36:19.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-79112-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:19.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[40592]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T08:36:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]': finished 2024-09-11T08:36:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[29355]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T08:36:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-79112-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:19.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:18 smithi043 ceph-mon[29355]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T08:36:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:18 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-37"}]': finished 2024-09-11T08:36:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:18 smithi172 ceph-mon[32987]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T08:36:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-79112-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:18 smithi172 ceph-mon[32987]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T08:36:20.382 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:19 smithi172 ceph-mon[32987]: pgmap v383: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:20.383 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:19 smithi172 ceph-mon[32987]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T08:36:20.383 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:20.383 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:20.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:19 smithi043 ceph-mon[40592]: pgmap v383: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:20.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:19 smithi043 ceph-mon[40592]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T08:36:20.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:20.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:20.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:19 smithi043 ceph-mon[29355]: pgmap v383: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:20.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:19 smithi043 ceph-mon[29355]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T08:36:20.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:20.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:21.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: pgmap v386: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: pgmap v386: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:21.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:21.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:21.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:21 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: pgmap v386: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:36:21.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:21.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:21 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:36:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[40592]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:36:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:22.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:22.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[29355]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:36:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:22 smithi172 ceph-mon[32987]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:36:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:22.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]: dispatch 2024-09-11T08:36:23.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:23] "GET /metrics HTTP/1.1" 200 230939 "" "Prometheus/2.43.0" 2024-09-11T08:36:23.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: pgmap v389: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:36:23.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]': finished 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]: dispatch 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]: dispatch 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: pgmap v389: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]': finished 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]: dispatch 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]: dispatch 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:23.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:23.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:23 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: pgmap v389: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:36:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-39"}]': finished 2024-09-11T08:36:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2774883959' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-79112-54"}]': finished 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]: dispatch 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]: dispatch 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:23.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:23 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:36:25.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]': finished 2024-09-11T08:36:25.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:25.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[29355]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:36:25.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:25.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]': finished 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[40592]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:25.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:25.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-39", "mode": "writeback"}]': finished 2024-09-11T08:36:25.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-79112-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:25.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-mon[32987]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:36:25.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:25.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:25.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:25.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:36:26.006 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:36:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:36:26.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: pgmap v392: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:26.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:26.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:36:26.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]': finished 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[40592]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: pgmap v392: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:26.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:26.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:26.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]': finished 2024-09-11T08:36:26.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:26 smithi043 ceph-mon[29355]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:36:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: pgmap v392: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:36:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:36:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]: dispatch 2024-09-11T08:36:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:36:26.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-79112-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:26.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-39"}]': finished 2024-09-11T08:36:26.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:26 smithi172 ceph-mon[32987]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:36:26.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-11T08:36:26.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-11T08:36:26.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-11T08:36:26.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: seed 79312 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (186 ms) 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (14929 ms) 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (68 ms) 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (15183 ms total) 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (10190 ms) 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-11T08:36:26.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8235 ms) 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10444 ms) 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [4] 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [5,4] 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [6,5,4] 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting some heads 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 7 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-11T08:36:26.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrubs... 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46928 ms) 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10445 ms) 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7555 ms) 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8042 ms) 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8020 ms) 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10042 ms) 2024-09-11T08:36:26.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8910 ms) 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10048 ms) 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12961 ms) 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7981 ms) 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8019 ms) 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13507 ms) 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7500 ms) 2024-09-11T08:36:26.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-11T08:36:26.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8022 ms) 2024-09-11T08:36:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[40592]: pgmap v395: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[40592]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:36:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[29355]: pgmap v395: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[29355]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:36:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:27 smithi172 ceph-mon[32987]: pgmap v395: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:27 smithi172 ceph-mon[32987]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:36:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:27 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:27.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[40592]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:36:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[29355]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:36:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:28 smithi172 ceph-mon[32987]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:36:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[40592]: pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[40592]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[29355]: pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[29355]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:29.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:29.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:29 smithi172 ceph-mon[32987]: pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:29 smithi172 ceph-mon[32987]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:36:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1144871259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]: dispatch 2024-09-11T08:36:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:29.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[40592]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-79112-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[29355]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:36:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:30.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-79112-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-79112-55"}]': finished 2024-09-11T08:36:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:30 smithi172 ceph-mon[32987]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:36:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:36:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:36:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:30.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-79112-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[40592]: pgmap v401: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:36:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:36:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-79112-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[40592]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:36:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-79112-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[29355]: pgmap v401: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:36:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:36:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-79112-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[29355]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:36:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-79112-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:31 smithi172 ceph-mon[32987]: pgmap v401: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:36:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:36:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-79112-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:31 smithi172 ceph-mon[32987]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:36:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-79112-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:31.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:31.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:36:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[40592]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:36:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:36:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:36:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[40592]: pgmap v404: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:36:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:36:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[29355]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:36:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:36:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:36:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[29355]: pgmap v404: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:36:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:32 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:32.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:36:32.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:32 smithi172 ceph-mon[32987]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:36:32.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:36:32.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:36:32.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:32 smithi172 ceph-mon[32987]: pgmap v404: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:36:32.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:32 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:33.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:36:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:33] "GET /metrics HTTP/1.1" 200 228531 "" "Prometheus/2.43.0" 2024-09-11T08:36:33.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-79112-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:33.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:36:33.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:33 smithi043 ceph-mon[40592]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:36:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-79112-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:36:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:33 smithi043 ceph-mon[29355]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:36:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-79112-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:36:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:33 smithi172 ceph-mon[32987]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:36:34.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[40592]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:36:34.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:34.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]: dispatch 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]: dispatch 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[40592]: pgmap v407: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[29355]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]: dispatch 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]: dispatch 2024-09-11T08:36:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:34 smithi043 ceph-mon[29355]: pgmap v407: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:34 smithi172 ceph-mon[32987]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:36:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]: dispatch 2024-09-11T08:36:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]: dispatch 2024-09-11T08:36:34.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:34 smithi172 ceph-mon[32987]: pgmap v407: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:36:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:36:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]': finished 2024-09-11T08:36:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:35 smithi172 ceph-mon[32987]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:36:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]': finished 2024-09-11T08:36:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:35 smithi043 ceph-mon[40592]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:36:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-41"}]': finished 2024-09-11T08:36:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:35 smithi043 ceph-mon[29355]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:36:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:36.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:36:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:36:36.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:36.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[29355]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:36:36.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:36.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:36.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[29355]: pgmap v410: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[40592]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:36:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:36.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:36 smithi043 ceph-mon[40592]: pgmap v410: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:36 smithi172 ceph-mon[32987]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:36:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]: dispatch 2024-09-11T08:36:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:36 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:36 smithi172 ceph-mon[32987]: pgmap v410: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-79312-6","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-79312-6","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:37.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2366735716' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-79112-56"}]': finished 2024-09-11T08:36:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:36:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-79112-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:37.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-79312-6","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:36:37.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:37.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:39.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:39 smithi043 ceph-mon[40592]: pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:39.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:39 smithi043 ceph-mon[29355]: pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:39 smithi172 ceph-mon[32987]: pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:40.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:36:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:36:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:36:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]': finished 2024-09-11T08:36:40.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:36:40.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:40.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:40.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:36:40.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:36:40.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:36:40.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:40.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]': finished 2024-09-11T08:36:40.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:36:40.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:40.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:36:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:36:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:36:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-79112-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_count","val": "8"}]': finished 2024-09-11T08:36:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:36:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:40.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:36:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[40592]: pgmap v416: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:36:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[40592]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:36:41.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:36:41.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:36:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[29355]: pgmap v416: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:36:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[29355]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:36:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:36:41.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:36:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:41 smithi172 ceph-mon[32987]: pgmap v416: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:36:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:41 smithi172 ceph-mon[32987]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:36:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:36:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:36:41.841 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-11T08:36:41.841 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-11T08:36:41.841 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: ok, hit_set contains 193:602f83fe:::foo:head 2024-09-11T08:36:41.841 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9048 ms) 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg_num = 32 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 0 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 1 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 2 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 3 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 4 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 5 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 6 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 7 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 8 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 9 ls 1726043802,0 2024-09-11T08:36:41.842 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 10 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 11 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 12 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 13 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 14 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 15 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 16 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 17 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 18 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 19 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 20 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 21 ls 1726043802,0 2024-09-11T08:36:41.843 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 22 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 23 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 24 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 25 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 26 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 27 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 28 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 29 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 30 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 31 ls 1726043802,0 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg_num = 32 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:6cac518f:::0:head 2024-09-11T08:36:41.844 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:02547ec2:::1:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f905c69b:::2:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:cfc208b3:::3:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d83876eb:::4:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b29083e3:::5:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c4fdafeb:::6:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:5c6b0b28:::7:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:bd63b0f1:::8:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:e960b815:::9:head 2024-09-11T08:36:41.845 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:52ea6a34:::10:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:89d3ae78:::11:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:de5d7c5f:::12:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:566253c9:::13:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:62a1935d:::14:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:863748b0:::15:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:3958e169:::16:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:4d4dabf9:::17:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:8391935d:::18:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:28883081:::19:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:69259c59:::20:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:4bdb80b7:::21:head 2024-09-11T08:36:41.846 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a11c5d71:::22:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:271af37b:::23:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:95b121be:::24:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:58d1031b:::25:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:0a050783:::26:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c709704c:::27:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:cbe56eaf:::28:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:86b4b162:::29:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:70d89383:::30:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:dd450c7c:::31:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:6d5729b1:::32:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c388f3fb:::33:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:56cfea31:::34:head 2024-09-11T08:36:41.847 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:9dbc1bf7:::35:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:40b74ccd:::36:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:4d5aaf42:::37:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:920f362c:::38:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:6cc53222:::39:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:9cad833f:::40:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:1ea84d41:::41:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c4480ef6:::42:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a694361e:::43:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d1bd33e9:::44:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:ddc2cd5d:::45:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:2b782207:::46:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:7b187fca:::47:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:90ecdf6f:::48:head 2024-09-11T08:36:41.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a5ed95fe:::49:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:ea0eaa55:::50:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f33ef17b:::51:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a0d1b2f6:::52:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:60c5229e:::53:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:edcbc575:::54:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:102cf253:::55:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:efb7fb0b:::56:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:50d0a326:::57:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d4dc5daf:::58:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:3a130462:::59:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:ec87ed71:::60:head 2024-09-11T08:36:41.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d5bc9454:::61:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:3ddfe313:::62:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:7c2816b9:::63:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:47e00e4d:::64:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c6410c18:::65:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b48ed237:::66:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:cd63ad31:::67:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b179e92b:::68:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:0d9f741a:::69:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:6d3352ae:::70:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c6d5c19e:::71:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:bc4729c3:::72:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:77e930b9:::73:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:0abeecfd:::74:head 2024-09-11T08:36:41.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b7c37e15:::75:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b6378398:::76:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:02bd68de:::77:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:cc795d2d:::78:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:630d4fea:::79:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:e0d29ef5:::80:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:fd6f13d2:::81:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:606461d5:::82:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:eadbdc43:::83:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:8761d0bb:::84:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:9ef0186f:::85:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:e0d41294:::86:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:961de695:::87:head 2024-09-11T08:36:41.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:1423148f:::88:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:633a8fa2:::89:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a8653809:::90:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:3dac8b33:::91:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:35aad435:::92:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f6dcc343:::93:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:dbbdad87:::94:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:1cb48ce0:::95:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:03cd461c:::96:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:17a4ea99:::97:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:9993c9a7:::98:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:6394211c:::99:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:94c7ae57:::100:head 2024-09-11T08:36:41.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:6fdee5bb:::101:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:9a477fd1:::102:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:eb850916:::103:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:affc56b9:::104:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b42dc814:::105:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f319f8f0:::106:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:9a40b9de:::107:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:8b524f28:::108:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:e3de589f:::109:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:90f90a5b:::110:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a7b4f1d7:::111:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:af51766e:::112:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b6f90bd1:::113:head 2024-09-11T08:36:41.853 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:e0261208:::114:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c9569ef7:::115:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:61bebe50:::116:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:fe93412b:::117:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d3d38bee:::118:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:3100ba0c:::119:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d0560ada:::120:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f0ea8b35:::121:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:766f231a:::122:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a07a2582:::123:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:bd7c6b3a:::124:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:fb2ddaff:::125:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:4408e1fe:::126:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:ee1df7a7:::127:head 2024-09-11T08:36:41.854 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c3002909:::128:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:4f48ffa9:::129:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:edf38733:::130:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c08425c0:::131:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:5f902d98:::132:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:41ea2c93:::133:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:813cee13:::134:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:0131818d:::135:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:26ba5a85:::136:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:381b8a5a:::137:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:28797e47:::138:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:bfca7f22:::139:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:36807075:::140:head 2024-09-11T08:36:41.855 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:80b03975:::141:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:5c15709b:::142:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f39ea15e:::143:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:ea992956:::144:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:48887b1c:::145:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:9f24a9dd:::146:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:987f100b:::147:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d2dd3581:::148:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:7fed1808:::149:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c80b70e9:::150:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:85ed90f9:::151:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:36428b24:::152:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d044c34a:::153:head 2024-09-11T08:36:41.856 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:7c18bf58:::154:head 2024-09-11T08:36:41.857 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d1c21232:::155:head 2024-09-11T08:36:41.857 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a7a3c575:::156:head 2024-09-11T08:36:41.857 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:87da0633:::157:head 2024-09-11T08:36:41.857 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d5ac3822:::158:head 2024-09-11T08:36:41.857 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:3f20522d:::159:head 2024-09-11T08:36:41.857 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:6ca26563:::160:head 2024-09-11T08:36:41.857 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:532ce135:::161:head 2024-09-11T08:36:41.857 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:c78863e6:::162:head 2024-09-11T08:36:42.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:42.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-11T08:36:42.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-79312-43","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]: dispatch 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]: dispatch 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-11T08:36:42.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:36:42.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:42.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:42.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-79312-43","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:36:42.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:42.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:42.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]: dispatch 2024-09-11T08:36:42.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]: dispatch 2024-09-11T08:36:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-11T08:36:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:36:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-79312-43","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:36:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:36:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]: dispatch 2024-09-11T08:36:42.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]: dispatch 2024-09-11T08:36:43.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:43.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]: dispatch 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]: dispatch 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]': finished 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]': finished 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]': finished 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]': finished 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:36:43.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]: dispatch 2024-09-11T08:36:43.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:43.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]': finished 2024-09-11T08:36:43.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]': finished 2024-09-11T08:36:43.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]': finished 2024-09-11T08:36:43.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]': finished 2024-09-11T08:36:43.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:36:43.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:43.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]: dispatch 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]: dispatch 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]: dispatch 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-43"}]': finished 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.11", "id": [1, 3]}]': finished 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.18", "id": [1, 3]}]': finished 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "195.1", "id": [4, 2]}]': finished 2024-09-11T08:36:43.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:36:43.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2824446094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:43.483 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]: dispatch 2024-09-11T08:36:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:36:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:43] "GET /metrics HTTP/1.1" 200 228519 "" "Prometheus/2.43.0" 2024-09-11T08:36:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[40592]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:36:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-79112-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[40592]: pgmap v422: 228 pgs: 3 peering, 225 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-09-11T08:36:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[29355]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:36:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-79112-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:44 smithi043 ceph-mon[29355]: pgmap v422: 228 pgs: 3 peering, 225 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-09-11T08:36:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-79112-57"}]': finished 2024-09-11T08:36:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:44 smithi172 ceph-mon[32987]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:36:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-79112-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:44.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:44 smithi172 ceph-mon[32987]: pgmap v422: 228 pgs: 3 peering, 225 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-09-11T08:36:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:36:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:36:46.080 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:36:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:36:46.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-79112-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:46.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:46 smithi043 ceph-mon[40592]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:36:46.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-79112-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:46.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-79112-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:46.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:46 smithi043 ceph-mon[29355]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:36:46.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-79112-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:46.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-79112-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:46.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:46 smithi172 ceph-mon[32987]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:36:46.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-79112-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:47.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:47.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:36:47.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: pgmap v425: 228 pgs: 32 unknown, 2 peering, 194 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-79112-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:36:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:47.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:47.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: pgmap v425: 228 pgs: 32 unknown, 2 peering, 194 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-79112-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:47.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:36:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: pgmap v425: 228 pgs: 32 unknown, 2 peering, 194 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:36:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-79112-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:36:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:47.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:36:48.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[40592]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:36:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:36:48.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:36:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[29355]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:36:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:36:48.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:36:48.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:48 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:48.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:36:48.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:48 smithi172 ceph-mon[32987]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:36:48.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:36:48.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:36:49.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[40592]: pgmap v428: 236 pgs: 40 unknown, 2 peering, 194 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:49.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:36:49.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[40592]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:36:49.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:36:49.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:49.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:36:49.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[29355]: pgmap v428: 236 pgs: 40 unknown, 2 peering, 194 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:49.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:36:49.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[29355]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:36:49.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:36:49.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:49.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:36:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:49 smithi172 ceph-mon[32987]: pgmap v428: 236 pgs: 40 unknown, 2 peering, 194 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:36:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:49 smithi172 ceph-mon[32987]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:36:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:36:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:49.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:36:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:36:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:50 smithi172 ceph-mon[32987]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:36:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:36:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:36:50.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:50 smithi172 ceph-mon[32987]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 21 B/s, 10 objects/s recovering 2024-09-11T08:36:51.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:51.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:36:51.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[40592]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:36:51.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[40592]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 21 B/s, 10 objects/s recovering 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[29355]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]: dispatch 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:36:51.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:50 smithi043 ceph-mon[29355]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 21 B/s, 10 objects/s recovering 2024-09-11T08:36:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:36:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:51 smithi172 ceph-mon[32987]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:36:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:36:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:36:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:51.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:52.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:52.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:36:52.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[40592]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:36:52.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/283198588' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-79112-58"}]': finished 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[29355]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:36:52.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:52.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:52.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:36:53.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:36:53.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[40592]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:36:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-79112-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[40592]: pgmap v434: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 21 B/s, 10 objects/s recovering 2024-09-11T08:36:53.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:36:53.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[29355]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:36:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-79112-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[29355]: pgmap v434: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 21 B/s, 10 objects/s recovering 2024-09-11T08:36:53.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:36:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:36:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:53 smithi172 ceph-mon[32987]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:36:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-79112-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:53 smithi172 ceph-mon[32987]: pgmap v434: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 21 B/s, 10 objects/s recovering 2024-09-11T08:36:53.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:53 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:36:54.097 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:36:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:53] "GET /metrics HTTP/1.1" 200 228519 "" "Prometheus/2.43.0" 2024-09-11T08:36:54.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:54 smithi043 ceph-mon[40592]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:36:54.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-79112-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:54.395 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:54 smithi043 ceph-mon[40592]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:36:54.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:54 smithi043 ceph-mon[29355]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:36:54.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-79112-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:54.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:54 smithi043 ceph-mon[29355]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:36:54.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:54 smithi172 ceph-mon[32987]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:36:54.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-79112-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:54.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:54 smithi172 ceph-mon[32987]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:36:55.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:55 smithi043 ceph-mon[40592]: pgmap v437: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:36:55.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:55 smithi043 ceph-mon[40592]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:36:55.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:55 smithi043 ceph-mon[29355]: pgmap v437: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:36:55.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:55 smithi043 ceph-mon[29355]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:36:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:55 smithi172 ceph-mon[32987]: pgmap v437: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:36:55.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:55 smithi172 ceph-mon[32987]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:36:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:36:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:36:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:36:56.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:36:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:36:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:56 smithi172 ceph-mon[32987]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:36:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:56 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:56 smithi172 ceph-mon[32987]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:36:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:56 smithi043 ceph-mon[40592]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:36:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:56 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:56 smithi043 ceph-mon[40592]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:36:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:56 smithi043 ceph-mon[29355]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:36:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:56 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:36:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:56 smithi043 ceph-mon[29355]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:36:57.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:7462ddf6:o.RoundTripAppendPP (3011 ms) 2024-09-11T08:36:57.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2983 ms) 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3009 ms) 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3005 ms) 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: setting pool EIO 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: max_success 2547, min_failed 2548 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4026 ms) 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3020 ms) 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (130751 ms total) 2024-09-11T08:36:57.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3006 ms) 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9022 ms) 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (15385 ms) 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (2559 ms) 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29973 ms total) 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-11T08:36:57.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3011 ms) 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3011 ms total) 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13991 ms) 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6990 ms) 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7006 ms) 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7012 ms) 2024-09-11T08:36:57.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3013 ms) 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7051 ms) 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6929 ms) 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7017 ms) 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7030 ms) 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7043 ms) 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7008 ms) 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-11T08:36:57.673 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7022 ms) 2024-09-11T08:36:57.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-11T08:36:57.674 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7015 ms) 2024-09-11T08:36:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:57 smithi172 ceph-mon[32987]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:36:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:57 smithi043 ceph-mon[40592]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:36:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:58.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:57 smithi043 ceph-mon[29355]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:36:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]: dispatch 2024-09-11T08:36:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:36:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:58 smithi172 ceph-mon[32987]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:36:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:58 smithi172 ceph-mon[32987]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:58 smithi043 ceph-mon[40592]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:36:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:58 smithi043 ceph-mon[40592]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:59.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2594487155' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-79112-59"}]': finished 2024-09-11T08:36:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:58 smithi043 ceph-mon[29355]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:36:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:58 smithi043 ceph-mon[29355]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:36:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:59 smithi172 ceph-mon[32987]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:36:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2060647261' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:36:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:00.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:59 smithi043 ceph-mon[40592]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:37:00.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2060647261' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:00.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:36:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:00.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:59 smithi043 ceph-mon[29355]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:37:00.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2060647261' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:00.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:36:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:00 smithi172 ceph-mon[32987]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:37:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:00 smithi172 ceph-mon[32987]: pgmap v446: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:37:01.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:01.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:00 smithi043 ceph-mon[40592]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:37:01.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:00 smithi043 ceph-mon[40592]: pgmap v446: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:37:01.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-79112-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:01.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:00 smithi043 ceph-mon[29355]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:37:01.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:00 smithi043 ceph-mon[29355]: pgmap v446: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:37:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:01 smithi172 ceph-mon[32987]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:37:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-79112-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:01 smithi043 ceph-mon[40592]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:37:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-79112-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:01 smithi043 ceph-mon[29355]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:37:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-79112-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-79112-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:02 smithi172 ceph-mon[32987]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:37:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-79112-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:02 smithi172 ceph-mon[32987]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:37:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-79112-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:02 smithi043 ceph-mon[40592]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:37:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-79112-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:02 smithi043 ceph-mon[40592]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:37:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-79112-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:03.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:02 smithi043 ceph-mon[29355]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:37:03.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-79112-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:03.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:02 smithi043 ceph-mon[29355]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:37:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:03 smithi172 ceph-mon[32987]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:37:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:03 smithi043 ceph-mon[40592]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:37:04.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:03 smithi043 ceph-mon[29355]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:37:04.145 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:37:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:03] "GET /metrics HTTP/1.1" 200 228565 "" "Prometheus/2.43.0" 2024-09-11T08:37:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-79112-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:37:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:04.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:04.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]: dispatch 2024-09-11T08:37:04.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]: dispatch 2024-09-11T08:37:04.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: pgmap v452: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:37:04.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]': finished 2024-09-11T08:37:04.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:04 smithi172 ceph-mon[32987]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-79112-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]: dispatch 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]: dispatch 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: pgmap v452: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]': finished 2024-09-11T08:37:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[40592]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-79112-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]: dispatch 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]: dispatch 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: pgmap v452: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-45"}]': finished 2024-09-11T08:37:05.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:04 smithi043 ceph-mon[29355]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:37:05.723 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:37:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:37:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:05 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:05 smithi172 ceph-mon[32987]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:37:05.982 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:37:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:37:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:05 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:05 smithi043 ceph-mon[29355]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:37:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:05 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:05 smithi043 ceph-mon[40592]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:37:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[40592]: pgmap v455: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[40592]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:37:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[29355]: pgmap v455: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:06 smithi043 ceph-mon[29355]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:37:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:06 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:06 smithi172 ceph-mon[32987]: pgmap v455: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:06 smithi172 ceph-mon[32987]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:37:08.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:08.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:08.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[29355]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:08.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:07 smithi043 ceph-mon[40592]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:37:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]: dispatch 2024-09-11T08:37:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:08.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/833601958' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-79112-61"}]': finished 2024-09-11T08:37:08.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:08.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:07 smithi172 ceph-mon[32987]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:37:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: pgmap v458: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: pgmap v458: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:09.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:09.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:09.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:37:09.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:09.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:09.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: pgmap v458: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:09.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:09.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:37:09.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:09.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:09.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:09.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]': finished 2024-09-11T08:37:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]: dispatch 2024-09-11T08:37:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:10 smithi172 ceph-mon[32987]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:37:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:10 smithi172 ceph-mon[32987]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:37:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]: dispatch 2024-09-11T08:37:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]': finished 2024-09-11T08:37:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]: dispatch 2024-09-11T08:37:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[40592]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:37:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[40592]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:37:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]: dispatch 2024-09-11T08:37:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-47"}]': finished 2024-09-11T08:37:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]: dispatch 2024-09-11T08:37:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[29355]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:37:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[29355]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:37:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]: dispatch 2024-09-11T08:37:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]': finished 2024-09-11T08:37:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[40592]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:37:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:12.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:12.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:12.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]': finished 2024-09-11T08:37:12.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[29355]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:37:12.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:12.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:12.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:11 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:12.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-79112-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:12.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-47", "mode": "writeback"}]': finished 2024-09-11T08:37:12.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:11 smithi172 ceph-mon[32987]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:37:12.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:12.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:37:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[40592]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:37:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[40592]: pgmap v464: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:37:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:37:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[29355]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:37:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:13.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:13.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[29355]: pgmap v464: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:37:13.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:13.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:13.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:12 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:37:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:12 smithi172 ceph-mon[32987]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:37:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:12 smithi172 ceph-mon[32987]: pgmap v464: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:37:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:12 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:13.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:12 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:13] "GET /metrics HTTP/1.1" 200 230954 "" "Prometheus/2.43.0" 2024-09-11T08:37:14.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[29355]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[40592]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:37:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:13 smithi172 ceph-mon[32987]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:37:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:37:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[40592]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:37:15.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:15.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:15.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:15.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:15.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:14 smithi043 ceph-mon[29355]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:37:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:37:15.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2633295475' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:15.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:15.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]: dispatch 2024-09-11T08:37:15.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:15.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:14 smithi172 ceph-mon[32987]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:37:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:37:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-79112-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-79112-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-79112-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:37:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-79112-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-79112-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-79112-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:37:16.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:37:16.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:37:16.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-79112-62"}]': finished 2024-09-11T08:37:16.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:37:16.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-79112-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-79112-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-79112-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:37:16.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:37:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:17 smithi172 ceph-mon[32987]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:37:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:17 smithi043 ceph-mon[40592]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:37:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:17 smithi043 ceph-mon[29355]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:37:18.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:18 smithi172 ceph-mon[32987]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:37:18.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:18.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:18.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-79112-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:18.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:18.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:18 smithi172 ceph-mon[32987]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:37:18.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:18.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[40592]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:37:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-79112-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[40592]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[29355]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-79112-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[29355]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:37:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:18.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:19 smithi172 ceph-mon[32987]: pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:19 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]': finished 2024-09-11T08:37:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:19 smithi172 ceph-mon[32987]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:37:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:19.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:19.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:19.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[40592]: pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]': finished 2024-09-11T08:37:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[40592]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:37:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[29355]: pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]': finished 2024-09-11T08:37:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[29355]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:37:19.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:19.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:19.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:19.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-47"}]: dispatch 2024-09-11T08:37:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:20 smithi043 ceph-mon[40592]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:37:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:20 smithi043 ceph-mon[40592]: pgmap v476: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:37:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:20 smithi043 ceph-mon[29355]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:37:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:21.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:20 smithi043 ceph-mon[29355]: pgmap v476: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:37:21.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:20 smithi172 ceph-mon[32987]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:37:21.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:21.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:20 smithi172 ceph-mon[32987]: pgmap v476: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:37:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[40592]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:37:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:22.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[29355]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:37:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:21 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:21 smithi172 ceph-mon[32987]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:37:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]: dispatch 2024-09-11T08:37:22.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:21 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[29355]: pgmap v479: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:23.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:23.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:23.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:23.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:22 smithi043 ceph-mon[40592]: pgmap v479: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:37:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1691078686' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-79112-63"}]': finished 2024-09-11T08:37:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:22 smithi172 ceph-mon[32987]: pgmap v479: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:37:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:37:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:23] "GET /metrics HTTP/1.1" 200 230954 "" "Prometheus/2.43.0" 2024-09-11T08:37:24.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:24.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:24.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:24 smithi172 ceph-mon[32987]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:37:24.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:24.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:24.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:24.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[40592]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:37:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-79112-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[29355]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:24.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:24.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]': finished 2024-09-11T08:37:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:37:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]': finished 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[40592]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]': finished 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:37:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:37:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:37:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]': finished 2024-09-11T08:37:25.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:25 smithi043 ceph-mon[29355]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:37:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:37:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-49"}]': finished 2024-09-11T08:37:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:37:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:37:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:37:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:37:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-79112-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-49", "mode": "readproxy"}]': finished 2024-09-11T08:37:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-mon[32987]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:37:26.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:37:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:37:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:26 smithi043 ceph-mon[40592]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:37:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:26 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:27.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:26 smithi043 ceph-mon[40592]: pgmap v485: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:37:27.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:26 smithi043 ceph-mon[29355]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:37:27.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:26 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:27.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:26 smithi043 ceph-mon[29355]: pgmap v485: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:37:27.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:26 smithi172 ceph-mon[32987]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:37:27.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:26 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:27.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:26 smithi172 ceph-mon[32987]: pgmap v485: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:37:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[40592]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:37:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:28.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:28.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[29355]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:37:28.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:28.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:28.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:27 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:27 smithi172 ceph-mon[32987]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:37:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[40592]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:37:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[40592]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[29355]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:37:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:28 smithi043 ceph-mon[29355]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:28 smithi172 ceph-mon[32987]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:37:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/81405919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:29.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]: dispatch 2024-09-11T08:37:29.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:28 smithi172 ceph-mon[32987]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[40592]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:37:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[29355]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:30.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-79112-64"}]': finished 2024-09-11T08:37:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:29 smithi172 ceph-mon[32987]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:37:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[40592]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:37:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[40592]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:37:31.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:31.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[29355]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:37:31.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:31.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:31.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:30 smithi043 ceph-mon[29355]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:37:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-79112-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:30 smithi172 ceph-mon[32987]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:37:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:30 smithi172 ceph-mon[32987]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:37:32.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:31 smithi043 ceph-mon[40592]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:37:32.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:31 smithi043 ceph-mon[29355]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:37:32.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:31 smithi172 ceph-mon[32987]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:37:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[40592]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:37:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[40592]: pgmap v494: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:37:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[40592]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:37:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[29355]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:37:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[29355]: pgmap v494: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:37:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:32 smithi043 ceph-mon[29355]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:37:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-79112-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:32 smithi172 ceph-mon[32987]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:37:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:32 smithi172 ceph-mon[32987]: pgmap v494: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:37:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:32 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:33.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:32 smithi172 ceph-mon[32987]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:37:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:37:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:33] "GET /metrics HTTP/1.1" 200 228543 "" "Prometheus/2.43.0" 2024-09-11T08:37:35.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:34 smithi043 ceph-mon[40592]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:37:35.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:35.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:35.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:34 smithi043 ceph-mon[40592]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:35.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:34 smithi043 ceph-mon[29355]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:37:35.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:35.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:35.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:34 smithi043 ceph-mon[29355]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:34 smithi172 ceph-mon[32987]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:37:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:35.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:34 smithi172 ceph-mon[32987]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:37:36.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:36.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:37:36.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:36.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:36.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:36.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:36.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:36.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:36.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[40592]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:37:36.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:36.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:37:36.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:36.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:36.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:36.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:36.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:36.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:36.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:35 smithi043 ceph-mon[29355]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:37:36.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:37:36.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:36.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:37:36.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1485488530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:36.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]: dispatch 2024-09-11T08:37:36.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:36.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:36.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-79112-65"}]': finished 2024-09-11T08:37:36.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:36.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:35 smithi172 ceph-mon[32987]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:37:37.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:37.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:37.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-79112-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]': finished 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-79112-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[40592]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:37.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:37.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:37.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-79112-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:37.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:37.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:37.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]': finished 2024-09-11T08:37:37.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-79112-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:37.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:36 smithi043 ceph-mon[29355]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:37:37.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:37.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:37.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:37.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:37.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:37.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-79112-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:37:37.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:37.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:37.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]': finished 2024-09-11T08:37:37.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-79112-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:37:37.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:36 smithi172 ceph-mon[32987]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:37:38.279 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFull::163:head 2024-09-11T08:37:38.279 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:5d165639:::164:head 2024-09-11T08:37:38.279 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f43765fc:::165:head 2024-09-11T08:37:38.279 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b4c720e9:::166:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:e694b040:::167:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:afa38db2:::168:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:77ba9f53:::169:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:87495034:::170:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:7c96bf0e:::171:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:dbe346cc:::172:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:e943ec24:::173:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f97a9c0c:::174:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:6f26e74d:::175:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:4f95e106:::176:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:0e6f2f8f:::177:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:05db05f1:::178:head 2024-09-11T08:37:38.280 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:38a78d66:::179:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:d095610b:::180:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:a1a9d709:::181:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:1e5d39db:::182:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:f7df4fb9:::183:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:03a7f161:::184:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:ba70721e:::185:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:28e5662d:::186:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:973d52de:::187:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:4303eb1c:::188:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:b990b48e:::189:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:29b8165b:::190:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:3547f197:::191:head 2024-09-11T08:37:38.281 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:7e260936:::192:head 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:1abec7b1:::193:head 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:10fdda93:::194:head 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:15817eea:::195:head 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:770bab57:::196:head 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:ed9e13e7:::197:head 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:71471a8f:::198:head 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 195:10fb1d02:::199:head 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (9021 ms) 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,0 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first is 1726043812 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,0 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,0 2024-09-11T08:37:38.282 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,1726043817,1726043818,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,1726043817,1726043818,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,1726043817,1726043818,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,1726043817,1726043818,1726043820,1726043821,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,1726043817,1726043818,1726043820,1726043821,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043812,1726043814,1726043815,1726043817,1726043818,1726043820,1726043821,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726043815,1726043817,1726043818,1726043820,1726043821,1726043823,1726043824,0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first now 1726043815, trimmed 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21086 ms) 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: foo0 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-11T08:37:38.283 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14041 ms) 2024-09-11T08:37:38.284 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-11T08:37:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-79112-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-79112-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-79112-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:38.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-49"}]: dispatch 2024-09-11T08:37:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[40592]: pgmap v503: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[40592]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:37:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-79112-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[40592]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:37:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[29355]: pgmap v503: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[29355]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:37:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-79112-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[29355]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:37:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:39 smithi172 ceph-mon[32987]: pgmap v503: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:39 smithi172 ceph-mon[32987]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:37:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-79112-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:39 smithi172 ceph-mon[32987]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:37:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:37:41.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:41.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[29355]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:37:41.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:41.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:41.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[29355]: pgmap v506: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:41.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:41.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[40592]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:37:41.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:41.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:41.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:40 smithi043 ceph-mon[40592]: pgmap v506: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:37:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:40 smithi172 ceph-mon[32987]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:37:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:37:41.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:40 smithi172 ceph-mon[32987]: pgmap v506: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:42.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:42.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:42.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[40592]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:37:42.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:42.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:42.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:42.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:42.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:42.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[29355]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:37:42.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:42.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:42.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:42.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:41 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:42.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:37:42.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:41 smithi172 ceph-mon[32987]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:37:42.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:42.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:42.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ PP2 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3029 ms) 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7031 ms) 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7024 ms) 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7018 ms) 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7046 ms) 2024-09-11T08:37:42.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-11T08:37:42.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7025 ms) 2024-09-11T08:37:42.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-11T08:37:42.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7016 ms) 2024-09-11T08:37:42.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139317 ms total) 2024-09-11T08:37:42.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-09-11T08:37:42.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-11T08:37:42.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (303053 ms total) 2024-09-11T08:37:42.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-11T08:37:43.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:43.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]': finished 2024-09-11T08:37:43.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:37:43.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:43.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]: dispatch 2024-09-11T08:37:43.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]: dispatch 2024-09-11T08:37:43.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: pgmap v509: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:43.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:43.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:43.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:43.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:43.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]': finished 2024-09-11T08:37:43.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[40592]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:37:43.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]': finished 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]: dispatch 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]: dispatch 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: pgmap v509: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:43.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:43.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:43.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]': finished 2024-09-11T08:37:43.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:42 smithi043 ceph-mon[29355]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:37:43.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:43.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-51"}]': finished 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]: dispatch 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]: dispatch 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]: dispatch 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: pgmap v509: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2973031376' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-79112-66"}]': finished 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-51", "mode": "writeback"}]': finished 2024-09-11T08:37:43.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:42 smithi172 ceph-mon[32987]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:37:44.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:44.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:44.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:44.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:44.145 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:37:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:43] "GET /metrics HTTP/1.1" 200 228547 "" "Prometheus/2.43.0" 2024-09-11T08:37:44.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:44.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:37:45.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:37:45.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[40592]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:37:45.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:45.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:45.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[40592]: pgmap v512: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:37:45.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:37:45.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[29355]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:37:45.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:45.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:45.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:44 smithi043 ceph-mon[29355]: pgmap v512: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:37:45.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:37:45.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:44 smithi172 ceph-mon[32987]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:37:45.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:45.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:37:45.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:44 smithi172 ceph-mon[32987]: pgmap v512: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:37:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:37:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:37:46.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:37:46.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:45 smithi043 ceph-mon[40592]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:37:46.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:46.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:46.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:37:46.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:45 smithi043 ceph-mon[29355]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:37:46.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:46.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:37:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:45 smithi172 ceph-mon[32987]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:37:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:46.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:37:46.232 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:37:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:37:47.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:47.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:47.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: pgmap v515: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[40592]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:37:47.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:37:47.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:47.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:47.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: pgmap v515: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:37:47.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:37:47.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:46 smithi043 ceph-mon[29355]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:37:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:37:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:37:47.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:37:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:37:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: pgmap v515: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:37:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:37:47.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:46 smithi172 ceph-mon[32987]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:37:48.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:37:48.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:37:48.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:48.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:37:48.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[40592]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:37:48.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:37:48.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:37:48.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:48.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:37:48.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:47 smithi043 ceph-mon[29355]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:37:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:37:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:37:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:47 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-51","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:37:48.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:47 smithi172 ceph-mon[32987]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:37:49.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:48 smithi172 ceph-mon[32987]: pgmap v518: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:49.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:48 smithi043 ceph-mon[40592]: pgmap v518: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:49.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:48 smithi043 ceph-mon[29355]: pgmap v518: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:50.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:49 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:37:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:49 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:37:50.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:49 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:37:51.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:50 smithi172 ceph-mon[32987]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 975 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:50 smithi043 ceph-mon[40592]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 975 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:51.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:50 smithi043 ceph-mon[29355]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 975 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:53 smithi172 ceph-mon[32987]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 819 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:53 smithi043 ceph-mon[40592]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 819 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:53 smithi043 ceph-mon[29355]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 819 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:37:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:53] "GET /metrics HTTP/1.1" 200 228547 "" "Prometheus/2.43.0" 2024-09-11T08:37:55.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:55 smithi172 ceph-mon[32987]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:55.481 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:37:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:37:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:37:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:55 smithi043 ceph-mon[40592]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:55 smithi043 ceph-mon[29355]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:56.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:37:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:37:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:57.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:57 smithi172 ceph-mon[32987]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:57 smithi043 ceph-mon[40592]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:57.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:37:57.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:57 smithi043 ceph-mon[29355]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:58 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:58 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:58 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:58 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:58 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:37:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:58 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:37:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:37:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[40592]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 999 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[40592]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:37:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[29355]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 999 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[29355]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:37:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:37:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:59 smithi172 ceph-mon[32987]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 999 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:37:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:37:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:59 smithi172 ceph-mon[32987]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:37:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:37:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:37:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:38:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]': finished 2024-09-11T08:38:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[40592]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:38:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:38:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:38:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]': finished 2024-09-11T08:38:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[29355]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:38:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:38:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:38:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]': finished 2024-09-11T08:38:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:00 smithi172 ceph-mon[32987]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:38:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:38:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-51"}]: dispatch 2024-09-11T08:38:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:01 smithi043 ceph-mon[40592]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:38:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:01 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:38:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:01 smithi043 ceph-mon[40592]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:38:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:01 smithi043 ceph-mon[29355]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:38:01.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:01 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:38:01.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:01 smithi043 ceph-mon[29355]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:38:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:01 smithi172 ceph-mon[32987]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:38:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:01 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:38:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:01 smithi172 ceph-mon[32987]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:38:02.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:02 smithi043 ceph-mon[40592]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:38:02.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:02.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:02.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:02 smithi043 ceph-mon[29355]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:38:02.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:02.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:02 smithi172 ceph-mon[32987]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:38:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[40592]: pgmap v529: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:38:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[40592]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:38:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-53"}]: dispatch 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-53"}]: dispatch 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[29355]: pgmap v529: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[29355]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-53"}]: dispatch 2024-09-11T08:38:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-53"}]: dispatch 2024-09-11T08:38:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:03 smithi172 ceph-mon[32987]: pgmap v529: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:38:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:03 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:03 smithi172 ceph-mon[32987]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:38:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-53"}]: dispatch 2024-09-11T08:38:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-53"}]: dispatch 2024-09-11T08:38:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:38:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:03] "GET /metrics HTTP/1.1" 200 228620 "" "Prometheus/2.43.0" 2024-09-11T08:38:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:04 smithi043 ceph-mon[40592]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:38:04.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:04 smithi043 ceph-mon[29355]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:38:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:04 smithi172 ceph-mon[32987]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:38:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:05 smithi043 ceph-mon[40592]: pgmap v532: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:05 smithi043 ceph-mon[40592]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:38:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:05.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:05.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:05 smithi043 ceph-mon[29355]: pgmap v532: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:05.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:05 smithi043 ceph-mon[29355]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:38:05.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:05.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:38:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:05 smithi172 ceph-mon[32987]: pgmap v532: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:05 smithi172 ceph-mon[32987]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:38:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:06.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:38:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:38:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[40592]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:38:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-55"}]: dispatch 2024-09-11T08:38:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-55"}]: dispatch 2024-09-11T08:38:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[29355]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:38:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-55"}]: dispatch 2024-09-11T08:38:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-55"}]: dispatch 2024-09-11T08:38:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:06 smithi172 ceph-mon[32987]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:38:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-55"}]: dispatch 2024-09-11T08:38:06.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-55"}]: dispatch 2024-09-11T08:38:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:07 smithi043 ceph-mon[40592]: pgmap v535: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:07 smithi043 ceph-mon[40592]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:38:07.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:07.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:07 smithi043 ceph-mon[29355]: pgmap v535: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:07.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:07 smithi043 ceph-mon[29355]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:38:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:07 smithi172 ceph-mon[32987]: pgmap v535: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:07 smithi172 ceph-mon[32987]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:38:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:08 smithi043 ceph-mon[40592]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:38:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:08 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:08 smithi043 ceph-mon[29355]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:38:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:08 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:08 smithi172 ceph-mon[32987]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:38:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:08 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[40592]: pgmap v538: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[40592]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:38:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-57"}]: dispatch 2024-09-11T08:38:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-57"}]: dispatch 2024-09-11T08:38:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[29355]: pgmap v538: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[29355]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:38:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-57"}]: dispatch 2024-09-11T08:38:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-57"}]: dispatch 2024-09-11T08:38:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:09 smithi172 ceph-mon[32987]: pgmap v538: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:09 smithi172 ceph-mon[32987]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:38:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-57"}]: dispatch 2024-09-11T08:38:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-57"}]: dispatch 2024-09-11T08:38:10.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:10 smithi043 ceph-mon[40592]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:38:10.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:10 smithi043 ceph-mon[29355]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:38:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:10 smithi172 ceph-mon[32987]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:38:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:11 smithi043 ceph-mon[40592]: pgmap v541: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:38:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:11 smithi043 ceph-mon[40592]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:38:11.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:11.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:11 smithi043 ceph-mon[29355]: pgmap v541: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:38:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:11 smithi043 ceph-mon[29355]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:38:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:11 smithi172 ceph-mon[32987]: pgmap v541: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:38:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:11 smithi172 ceph-mon[32987]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:38:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:38:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-59"}]: dispatch 2024-09-11T08:38:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-59"}]: dispatch 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-59"}]: dispatch 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-59"}]: dispatch 2024-09-11T08:38:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:38:12.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:38:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-59"}]: dispatch 2024-09-11T08:38:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-59"}]: dispatch 2024-09-11T08:38:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:38:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:13 smithi043 ceph-mon[40592]: pgmap v544: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:38:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:13 smithi043 ceph-mon[40592]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:38:13.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:13 smithi043 ceph-mon[29355]: pgmap v544: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:38:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:13 smithi043 ceph-mon[29355]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:38:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:13 smithi172 ceph-mon[32987]: pgmap v544: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:38:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:13 smithi172 ceph-mon[32987]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:38:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:38:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:13] "GET /metrics HTTP/1.1" 200 228604 "" "Prometheus/2.43.0" 2024-09-11T08:38:14.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:14 smithi043 ceph-mon[40592]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:38:14.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:14.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:14.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:14 smithi043 ceph-mon[29355]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:38:14.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:14.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:14 smithi172 ceph-mon[32987]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:38:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[40592]: pgmap v547: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:38:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[40592]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[29355]: pgmap v547: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[29355]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-mon[32987]: pgmap v547: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:38:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-mon[32987]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:38:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:16.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:38:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:38:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[40592]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:38:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-61"}]: dispatch 2024-09-11T08:38:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-61"}]: dispatch 2024-09-11T08:38:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[29355]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:38:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-61"}]: dispatch 2024-09-11T08:38:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-61"}]: dispatch 2024-09-11T08:38:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:16 smithi172 ceph-mon[32987]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:38:16.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:16.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:16.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-61"}]: dispatch 2024-09-11T08:38:16.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-61"}]: dispatch 2024-09-11T08:38:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:17 smithi043 ceph-mon[40592]: pgmap v550: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:38:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:17 smithi043 ceph-mon[40592]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:38:17.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:17.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:17 smithi043 ceph-mon[29355]: pgmap v550: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:38:17.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:17 smithi043 ceph-mon[29355]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:38:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:17 smithi172 ceph-mon[32987]: pgmap v550: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:38:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:17 smithi172 ceph-mon[32987]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:38:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[40592]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:38:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[40592]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:38:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[29355]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:38:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:18 smithi043 ceph-mon[29355]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:38:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:18 smithi172 ceph-mon[32987]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:38:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:18.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:18.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:18 smithi172 ceph-mon[32987]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:38:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[40592]: pgmap v553: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[40592]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:38:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[29355]: pgmap v553: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:19.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:19.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:19 smithi043 ceph-mon[29355]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:38:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:19 smithi172 ceph-mon[32987]: pgmap v553: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:19 smithi172 ceph-mon[32987]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:38:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:20 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:20 smithi172 ceph-mon[32987]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:38:20.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:20 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:20.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:20 smithi043 ceph-mon[40592]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:38:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:20 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:20 smithi043 ceph-mon[29355]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:38:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:21 smithi172 ceph-mon[32987]: pgmap v556: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:21 smithi172 ceph-mon[32987]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:38:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:21 smithi043 ceph-mon[40592]: pgmap v556: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:21 smithi043 ceph-mon[40592]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:38:21.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:21 smithi043 ceph-mon[29355]: pgmap v556: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:21.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:21 smithi043 ceph-mon[29355]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:38:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:22 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:22 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:22 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:22 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:38:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:22 smithi172 ceph-mon[32987]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:38:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:38:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[40592]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:38:22.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:38:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:22 smithi043 ceph-mon[29355]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:38:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:23 smithi172 ceph-mon[32987]: pgmap v559: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:23 smithi172 ceph-mon[32987]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T08:38:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:23 smithi043 ceph-mon[40592]: pgmap v559: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:23 smithi043 ceph-mon[40592]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T08:38:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:23 smithi043 ceph-mon[29355]: pgmap v559: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:38:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:23 smithi043 ceph-mon[29355]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T08:38:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:38:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:23] "GET /metrics HTTP/1.1" 200 228604 "" "Prometheus/2.43.0" 2024-09-11T08:38:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:25 smithi172 ceph-mon[32987]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-11T08:38:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:38:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:25 smithi043 ceph-mon[40592]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-11T08:38:25.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:25 smithi043 ceph-mon[29355]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-11T08:38:26.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:38:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:38:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:27 smithi172 ceph-mon[32987]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-11T08:38:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:27 smithi043 ceph-mon[40592]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-11T08:38:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:27 smithi043 ceph-mon[29355]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-11T08:38:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:29 smithi172 ceph-mon[32987]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-11T08:38:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:29 smithi043 ceph-mon[40592]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-11T08:38:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:29 smithi043 ceph-mon[29355]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-11T08:38:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:31 smithi172 ceph-mon[32987]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-11T08:38:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:31 smithi043 ceph-mon[40592]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-11T08:38:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:31 smithi043 ceph-mon[29355]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-11T08:38:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:33 smithi172 ceph-mon[32987]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-11T08:38:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:33 smithi172 ceph-mon[32987]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T08:38:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:33 smithi043 ceph-mon[40592]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-11T08:38:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:33 smithi043 ceph-mon[40592]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T08:38:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:33 smithi043 ceph-mon[29355]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-11T08:38:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:33 smithi043 ceph-mon[29355]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T08:38:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:38:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:33] "GET /metrics HTTP/1.1" 200 228665 "" "Prometheus/2.43.0" 2024-09-11T08:38:35.045 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:34 smithi043 ceph-mon[40592]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T08:38:35.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:34 smithi043 ceph-mon[40592]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:35.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:34 smithi043 ceph-mon[29355]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T08:38:35.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:34 smithi043 ceph-mon[29355]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:34 smithi172 ceph-mon[32987]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T08:38:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:34 smithi172 ceph-mon[32987]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:38:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:36.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:38:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:38:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:37 smithi172 ceph-mon[32987]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:37 smithi043 ceph-mon[40592]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:37 smithi043 ceph-mon[29355]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:39 smithi172 ceph-mon[32987]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:38:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:39 smithi043 ceph-mon[40592]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:38:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:39 smithi043 ceph-mon[29355]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:38:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:41 smithi172 ceph-mon[32987]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:41 smithi043 ceph-mon[40592]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:41 smithi043 ceph-mon[29355]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:43 smithi172 ceph-mon[32987]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:43.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:43 smithi172 ceph-mon[32987]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T08:38:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:43 smithi043 ceph-mon[40592]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:43 smithi043 ceph-mon[40592]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T08:38:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:43 smithi043 ceph-mon[29355]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:43 smithi043 ceph-mon[29355]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T08:38:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:38:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:43] "GET /metrics HTTP/1.1" 200 228643 "" "Prometheus/2.43.0" 2024-09-11T08:38:45.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:45 smithi172 ceph-mon[32987]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:45.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:45 smithi172 ceph-mon[32987]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T08:38:45.481 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:38:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:45 smithi043 ceph-mon[40592]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:45 smithi043 ceph-mon[40592]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T08:38:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:45 smithi043 ceph-mon[29355]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:45 smithi043 ceph-mon[29355]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T08:38:46.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:38:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:38:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:47.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:47 smithi172 ceph-mon[32987]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:47 smithi043 ceph-mon[29355]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:47 smithi043 ceph-mon[40592]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:38:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:49 smithi172 ceph-mon[32987]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:38:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:49 smithi043 ceph-mon[29355]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:38:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:49 smithi043 ceph-mon[40592]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:38:51.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:51 smithi172 ceph-mon[32987]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:38:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:51 smithi043 ceph-mon[29355]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:38:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:51 smithi043 ceph-mon[40592]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:38:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:53 smithi172 ceph-mon[32987]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:53.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:53 smithi172 ceph-mon[32987]: osdmap e394: 8 total, 8 up, 8 in 2024-09-11T08:38:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:53 smithi043 ceph-mon[29355]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:53 smithi043 ceph-mon[29355]: osdmap e394: 8 total, 8 up, 8 in 2024-09-11T08:38:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:53 smithi043 ceph-mon[40592]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:53.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:53 smithi043 ceph-mon[40592]: osdmap e394: 8 total, 8 up, 8 in 2024-09-11T08:38:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:38:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:53] "GET /metrics HTTP/1.1" 200 228643 "" "Prometheus/2.43.0" 2024-09-11T08:38:55.481 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:38:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:38:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:55.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:55 smithi172 ceph-mon[32987]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:38:55.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:55.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:55.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-63"}]: dispatch 2024-09-11T08:38:55.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-63"}]: dispatch 2024-09-11T08:38:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[29355]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:38:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-63"}]: dispatch 2024-09-11T08:38:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-63"}]: dispatch 2024-09-11T08:38:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[40592]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:38:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:38:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-63"}]: dispatch 2024-09-11T08:38:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-63"}]: dispatch 2024-09-11T08:38:56.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:38:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:38:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:56 smithi043 ceph-mon[40592]: osdmap e395: 8 total, 8 up, 8 in 2024-09-11T08:38:56.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:56 smithi043 ceph-mon[29355]: osdmap e395: 8 total, 8 up, 8 in 2024-09-11T08:38:56.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:56 smithi172 ceph-mon[32987]: osdmap e395: 8 total, 8 up, 8 in 2024-09-11T08:38:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[40592]: pgmap v584: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[40592]: osdmap e396: 8 total, 8 up, 8 in 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[29355]: pgmap v584: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[29355]: osdmap e396: 8 total, 8 up, 8 in 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:57 smithi172 ceph-mon[32987]: pgmap v584: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:57 smithi172 ceph-mon[32987]: osdmap e396: 8 total, 8 up, 8 in 2024-09-11T08:38:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:38:57.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[40592]: osdmap e397: 8 total, 8 up, 8 in 2024-09-11T08:38:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[29355]: osdmap e397: 8 total, 8 up, 8 in 2024-09-11T08:38:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:38:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:58 smithi172 ceph-mon[32987]: osdmap e397: 8 total, 8 up, 8 in 2024-09-11T08:38:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:38:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:38:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:59 smithi043 ceph-mon[29355]: pgmap v587: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:38:59 smithi043 ceph-mon[29355]: osdmap e398: 8 total, 8 up, 8 in 2024-09-11T08:38:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:59 smithi043 ceph-mon[40592]: pgmap v587: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:38:59 smithi043 ceph-mon[40592]: osdmap e398: 8 total, 8 up, 8 in 2024-09-11T08:38:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:59 smithi172 ceph-mon[32987]: pgmap v587: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:38:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:38:59 smithi172 ceph-mon[32987]: osdmap e398: 8 total, 8 up, 8 in 2024-09-11T08:39:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:00 smithi043 ceph-mon[40592]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:39:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:00 smithi043 ceph-mon[29355]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:39:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:00 smithi172 ceph-mon[32987]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:39:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:01 smithi043 ceph-mon[40592]: pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:39:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:01 smithi043 ceph-mon[40592]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:39:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:01 smithi043 ceph-mon[29355]: pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:39:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:01 smithi043 ceph-mon[29355]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:39:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:01 smithi172 ceph-mon[32987]: pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:39:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:01 smithi172 ceph-mon[32987]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:39:02.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:02 smithi043 ceph-mon[40592]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:39:02.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:02 smithi043 ceph-mon[29355]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:39:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:02 smithi172 ceph-mon[32987]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:39:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:03 smithi043 ceph-mon[40592]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:39:03.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:03 smithi043 ceph-mon[29355]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:39:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:03 smithi172 ceph-mon[32987]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:39:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:39:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:03] "GET /metrics HTTP/1.1" 200 228561 "" "Prometheus/2.43.0" 2024-09-11T08:39:05.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:05 smithi043 ceph-mon[40592]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-11T08:39:05.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:05 smithi043 ceph-mon[29355]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-11T08:39:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:05 smithi172 ceph-mon[32987]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-11T08:39:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:39:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:06.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:39:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:39:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:07 smithi043 ceph-mon[40592]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-11T08:39:07.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:07.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:07 smithi043 ceph-mon[29355]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-11T08:39:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:07 smithi172 ceph-mon[32987]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-11T08:39:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:08 smithi043 ceph-mon[40592]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:39:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:08 smithi043 ceph-mon[40592]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 909 B/s wr, 4 op/s 2024-09-11T08:39:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:08 smithi043 ceph-mon[29355]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:39:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:08 smithi043 ceph-mon[29355]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 909 B/s wr, 4 op/s 2024-09-11T08:39:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:08 smithi172 ceph-mon[32987]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:39:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:08 smithi172 ceph-mon[32987]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 909 B/s wr, 4 op/s 2024-09-11T08:39:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:11 smithi172 ceph-mon[32987]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 806 B/s wr, 4 op/s 2024-09-11T08:39:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:11 smithi043 ceph-mon[40592]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 806 B/s wr, 4 op/s 2024-09-11T08:39:11.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:11 smithi043 ceph-mon[29355]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 806 B/s wr, 4 op/s 2024-09-11T08:39:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-65"}]: dispatch 2024-09-11T08:39:12.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-65"}]: dispatch 2024-09-11T08:39:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-65"}]: dispatch 2024-09-11T08:39:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-65"}]: dispatch 2024-09-11T08:39:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-65"}]: dispatch 2024-09-11T08:39:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-65"}]: dispatch 2024-09-11T08:39:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:13 smithi172 ceph-mon[32987]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:39:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:13 smithi172 ceph-mon[32987]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:39:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:13 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:13 smithi043 ceph-mon[40592]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:39:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:13 smithi043 ceph-mon[40592]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:39:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:13 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:13 smithi043 ceph-mon[29355]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:39:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:13 smithi043 ceph-mon[29355]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:39:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:13 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:39:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:13] "GET /metrics HTTP/1.1" 200 226149 "" "Prometheus/2.43.0" 2024-09-11T08:39:14.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[29355]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:39:14.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:14.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:14.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:14.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[29355]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:39:14.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:14.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[40592]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:39:14.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:14.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:14.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:14.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[40592]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:39:14.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:14 smithi172 ceph-mon[32987]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:39:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:14 smithi172 ceph-mon[32987]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:39:14.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:15 smithi043 ceph-mon[40592]: pgmap v602: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:15 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:15 smithi043 ceph-mon[40592]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:39:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:15 smithi043 ceph-mon[29355]: pgmap v602: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:15 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:15 smithi043 ceph-mon[29355]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:39:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:39:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:15 smithi172 ceph-mon[32987]: pgmap v602: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:15 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:15 smithi172 ceph-mon[32987]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:39:16.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:39:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:39:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-67"}]: dispatch 2024-09-11T08:39:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-67"}]: dispatch 2024-09-11T08:39:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-67"}]: dispatch 2024-09-11T08:39:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-67"}]: dispatch 2024-09-11T08:39:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-67"}]: dispatch 2024-09-11T08:39:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-67"}]: dispatch 2024-09-11T08:39:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:17 smithi043 ceph-mon[40592]: pgmap v605: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:17 smithi043 ceph-mon[40592]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:39:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:17 smithi043 ceph-mon[29355]: pgmap v605: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:17 smithi043 ceph-mon[29355]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:39:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:17 smithi172 ceph-mon[32987]: pgmap v605: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:17 smithi172 ceph-mon[32987]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:39:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[40592]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:39:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[40592]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:39:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[29355]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:39:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:18.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:18 smithi043 ceph-mon[29355]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:39:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:18 smithi172 ceph-mon[32987]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:39:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:18.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:18 smithi172 ceph-mon[32987]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:39:19.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[29355]: pgmap v608: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:19.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:19.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:19.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:19.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-69"}]: dispatch 2024-09-11T08:39:19.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-69"}]: dispatch 2024-09-11T08:39:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[40592]: pgmap v608: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-69"}]: dispatch 2024-09-11T08:39:19.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-69"}]: dispatch 2024-09-11T08:39:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:19 smithi172 ceph-mon[32987]: pgmap v608: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-69"}]: dispatch 2024-09-11T08:39:19.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-69"}]: dispatch 2024-09-11T08:39:20.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:20 smithi043 ceph-mon[40592]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:39:20.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:20 smithi043 ceph-mon[29355]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:39:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:20 smithi172 ceph-mon[32987]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:39:21.632 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:21 smithi172 ceph-mon[32987]: pgmap v611: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:39:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:21 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:21 smithi172 ceph-mon[32987]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:39:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[40592]: pgmap v611: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:39:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[40592]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:39:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:21.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[29355]: pgmap v611: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:39:21.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:21.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[29355]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:39:21.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:21.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[29355]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:39:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:39:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:39:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[40592]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:39:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:39:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:39:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:22 smithi172 ceph-mon[32987]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:39:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:39:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:39:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[40592]: pgmap v614: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:39:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[40592]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[29355]: pgmap v614: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:39:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:39:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[29355]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:39:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:23 smithi172 ceph-mon[32987]: pgmap v614: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:39:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:39:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:23 smithi172 ceph-mon[32987]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:39:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:39:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:23] "GET /metrics HTTP/1.1" 200 226149 "" "Prometheus/2.43.0" 2024-09-11T08:39:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:24 smithi043 ceph-mon[40592]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:39:24.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:24 smithi043 ceph-mon[29355]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:39:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:24 smithi172 ceph-mon[32987]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:39:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:25 smithi043 ceph-mon[40592]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-11T08:39:25.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:25 smithi043 ceph-mon[29355]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-11T08:39:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:25 smithi172 ceph-mon[32987]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-11T08:39:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:39:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:26.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:39:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:39:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:27 smithi043 ceph-mon[40592]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-11T08:39:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:27 smithi043 ceph-mon[29355]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-11T08:39:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:27 smithi172 ceph-mon[32987]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-11T08:39:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:29 smithi043 ceph-mon[40592]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 893 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:39:29.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:29 smithi043 ceph-mon[29355]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 893 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:39:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:29 smithi172 ceph-mon[32987]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 893 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:39:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:31 smithi043 ceph-mon[40592]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-11T08:39:31.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:31 smithi043 ceph-mon[29355]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-11T08:39:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:31 smithi172 ceph-mon[32987]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-11T08:39:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:33 smithi043 ceph-mon[40592]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-11T08:39:33.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:33 smithi043 ceph-mon[29355]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-11T08:39:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:33 smithi172 ceph-mon[32987]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-11T08:39:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:39:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:33] "GET /metrics HTTP/1.1" 200 228677 "" "Prometheus/2.43.0" 2024-09-11T08:39:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:35 smithi043 ceph-mon[40592]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 565 B/s wr, 2 op/s 2024-09-11T08:39:35.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:35 smithi043 ceph-mon[29355]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 565 B/s wr, 2 op/s 2024-09-11T08:39:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:35 smithi172 ceph-mon[32987]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 565 B/s wr, 2 op/s 2024-09-11T08:39:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:39:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:36.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:39:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:39:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:37 smithi043 ceph-mon[40592]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-11T08:39:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:37 smithi043 ceph-mon[29355]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-11T08:39:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:37 smithi172 ceph-mon[32987]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-11T08:39:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:39 smithi043 ceph-mon[29355]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-11T08:39:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:39 smithi043 ceph-mon[40592]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-11T08:39:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:39 smithi172 ceph-mon[32987]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-11T08:39:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:41 smithi043 ceph-mon[29355]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-11T08:39:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:41 smithi043 ceph-mon[40592]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-11T08:39:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:41 smithi172 ceph-mon[32987]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-11T08:39:42.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:42.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:42.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:43 smithi043 ceph-mon[40592]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:39:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:43 smithi043 ceph-mon[29355]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:39:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:43 smithi172 ceph-mon[32987]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:39:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:39:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:43] "GET /metrics HTTP/1.1" 200 228629 "" "Prometheus/2.43.0" 2024-09-11T08:39:44.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:44.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:44.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-71"}]: dispatch 2024-09-11T08:39:44.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-71"}]: dispatch 2024-09-11T08:39:44.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:44.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:44.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-71"}]: dispatch 2024-09-11T08:39:44.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-71"}]: dispatch 2024-09-11T08:39:44.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:44.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:44.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-71"}]: dispatch 2024-09-11T08:39:44.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-71"}]: dispatch 2024-09-11T08:39:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:45 smithi043 ceph-mon[40592]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:39:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:45 smithi043 ceph-mon[40592]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:39:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:45 smithi043 ceph-mon[29355]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:39:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:45 smithi043 ceph-mon[29355]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:39:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:45 smithi172 ceph-mon[32987]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:39:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:45 smithi172 ceph-mon[32987]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:39:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:39:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:46.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:39:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:39:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:46 smithi043 ceph-mon[40592]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:39:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:46 smithi043 ceph-mon[29355]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:39:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:46 smithi172 ceph-mon[32987]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:39:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:39:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[40592]: pgmap v630: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[40592]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:39:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[29355]: pgmap v630: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[29355]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:39:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:47 smithi172 ceph-mon[32987]: pgmap v630: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:47 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:39:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:47 smithi172 ceph-mon[32987]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:39:47.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:39:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:48 smithi043 ceph-mon[40592]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:39:48.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:48 smithi043 ceph-mon[29355]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:39:48.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:48 smithi172 ceph-mon[32987]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:39:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:49 smithi043 ceph-mon[40592]: pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:49 smithi043 ceph-mon[40592]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:39:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:49 smithi043 ceph-mon[29355]: pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:49 smithi043 ceph-mon[29355]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:39:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:49 smithi172 ceph-mon[32987]: pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:49 smithi172 ceph-mon[32987]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:39:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:51 smithi043 ceph-mon[40592]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:39:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:51 smithi043 ceph-mon[29355]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:39:51.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:51 smithi172 ceph-mon[32987]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:39:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:53 smithi043 ceph-mon[40592]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 1 op/s 2024-09-11T08:39:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:53 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:53 smithi043 ceph-mon[29355]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 1 op/s 2024-09-11T08:39:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:53 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:53 smithi172 ceph-mon[32987]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 1 op/s 2024-09-11T08:39:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:53 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:39:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:39:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:53] "GET /metrics HTTP/1.1" 200 228629 "" "Prometheus/2.43.0" 2024-09-11T08:39:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:55 smithi043 ceph-mon[40592]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 916 B/s wr, 4 op/s 2024-09-11T08:39:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:55 smithi043 ceph-mon[29355]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 916 B/s wr, 4 op/s 2024-09-11T08:39:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:55 smithi172 ceph-mon[32987]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 916 B/s wr, 4 op/s 2024-09-11T08:39:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:39:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:39:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:56.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:39:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:39:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:57 smithi043 ceph-mon[40592]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 813 B/s wr, 3 op/s 2024-09-11T08:39:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:57 smithi043 ceph-mon[29355]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 813 B/s wr, 3 op/s 2024-09-11T08:39:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:57 smithi172 ceph-mon[32987]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 813 B/s wr, 3 op/s 2024-09-11T08:39:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[29355]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:39:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-73"}]: dispatch 2024-09-11T08:39:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-73"}]: dispatch 2024-09-11T08:39:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[40592]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:39:59.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:59.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:59.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-73"}]: dispatch 2024-09-11T08:39:59.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:39:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-73"}]: dispatch 2024-09-11T08:39:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:59 smithi172 ceph-mon[32987]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:39:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:39:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-73"}]: dispatch 2024-09-11T08:39:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:39:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-73"}]: dispatch 2024-09-11T08:40:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[40592]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:40:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[40592]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T08:40:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[40592]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:40:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[40592]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:40:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[40592]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:40:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[29355]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:40:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[29355]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T08:40:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[29355]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:40:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[29355]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:40:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:00 smithi043 ceph-mon[29355]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:40:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:00 smithi172 ceph-mon[32987]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:40:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:00 smithi172 ceph-mon[32987]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T08:40:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:00 smithi172 ceph-mon[32987]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:40:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:00 smithi172 ceph-mon[32987]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:40:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:00 smithi172 ceph-mon[32987]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:40:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:01 smithi172 ceph-mon[32987]: pgmap v641: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:01 smithi172 ceph-mon[32987]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T08:40:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:01 smithi043 ceph-mon[40592]: pgmap v641: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:01 smithi043 ceph-mon[40592]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T08:40:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:01 smithi043 ceph-mon[29355]: pgmap v641: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:01 smithi043 ceph-mon[29355]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T08:40:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:01.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:02 smithi172 ceph-mon[32987]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T08:40:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:02 smithi043 ceph-mon[40592]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T08:40:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:02 smithi043 ceph-mon[29355]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T08:40:02.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:03 smithi172 ceph-mon[32987]: pgmap v644: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:03 smithi172 ceph-mon[32987]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T08:40:03.732 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:40:03 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:40:03.441+0000 7fa614671640 -1 snap_mapper.add_oid found existing snaps mapped on 63:af3d1ce0:test-rados-api-smithi043-79312-76::foo:28, removing 2024-09-11T08:40:03.732 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:40:03 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:40:03.441+0000 7f6b1784e640 -1 snap_mapper.add_oid found existing snaps mapped on 63:af3d1ce0:test-rados-api-smithi043-79312-76::foo:28, removing 2024-09-11T08:40:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:03 smithi043 ceph-mon[40592]: pgmap v644: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:03 smithi043 ceph-mon[40592]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T08:40:03.734 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:40:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:40:03.441+0000 7f5aba41e640 -1 snap_mapper.add_oid found existing snaps mapped on 63:af3d1ce0:test-rados-api-smithi043-79312-76::foo:28, removing 2024-09-11T08:40:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:03 smithi043 ceph-mon[29355]: pgmap v644: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:03 smithi043 ceph-mon[29355]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T08:40:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:40:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:03] "GET /metrics HTTP/1.1" 200 228623 "" "Prometheus/2.43.0" 2024-09-11T08:40:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:04 smithi172 ceph-mon[32987]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T08:40:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-75"}]: dispatch 2024-09-11T08:40:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-75"}]: dispatch 2024-09-11T08:40:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[29355]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T08:40:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-75"}]: dispatch 2024-09-11T08:40:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-75"}]: dispatch 2024-09-11T08:40:04.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[40592]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T08:40:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-75"}]: dispatch 2024-09-11T08:40:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-75"}]: dispatch 2024-09-11T08:40:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:05 smithi172 ceph-mon[32987]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:40:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:05 smithi172 ceph-mon[32987]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T08:40:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:40:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:05.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:05 smithi043 ceph-mon[40592]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:40:05.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:05 smithi043 ceph-mon[40592]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T08:40:05.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:05 smithi043 ceph-mon[29355]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:40:05.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:05 smithi043 ceph-mon[29355]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T08:40:06.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:40:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:40:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:06 smithi172 ceph-mon[32987]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T08:40:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:06 smithi172 ceph-mon[32987]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T08:40:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:06.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[29355]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T08:40:06.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:06.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:06.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[29355]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T08:40:06.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[40592]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T08:40:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[40592]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T08:40:06.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:07.731 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:40:07 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:40:07.453+0000 7f5d96a2a640 -1 snap_mapper.add_oid found existing snaps mapped on 63:5e18cd9e:test-rados-api-smithi043-79312-78::foo:29, removing 2024-09-11T08:40:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:07 smithi172 ceph-mon[32987]: pgmap v650: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:40:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:07 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:07 smithi172 ceph-mon[32987]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T08:40:07.894 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:40:07.453+0000 7f5abfc29640 -1 snap_mapper.add_oid found existing snaps mapped on 63:5e18cd9e:test-rados-api-smithi043-79312-78::foo:29, removing 2024-09-11T08:40:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-mon[29355]: pgmap v650: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:40:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-mon[29355]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T08:40:07.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:07.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-mon[40592]: pgmap v650: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:40:07.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:07.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-mon[40592]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T08:40:07.896 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:40:07 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2[56903]: 2024-09-11T08:40:07.452+0000 7f5cedab0640 -1 snap_mapper.add_oid found existing snaps mapped on 63:5e18cd9e:test-rados-api-smithi043-79312-78::foo:29, removing 2024-09-11T08:40:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-77"}]: dispatch 2024-09-11T08:40:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-77"}]: dispatch 2024-09-11T08:40:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-77"}]: dispatch 2024-09-11T08:40:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-77"}]: dispatch 2024-09-11T08:40:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-77"}]: dispatch 2024-09-11T08:40:08.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-77"}]: dispatch 2024-09-11T08:40:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:09 smithi172 ceph-mon[32987]: pgmap v653: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:09 smithi172 ceph-mon[32987]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T08:40:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:09 smithi043 ceph-mon[40592]: pgmap v653: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:09 smithi043 ceph-mon[40592]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T08:40:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:09 smithi043 ceph-mon[29355]: pgmap v653: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:09 smithi043 ceph-mon[29355]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T08:40:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:10 smithi172 ceph-mon[32987]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T08:40:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:10.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:10 smithi043 ceph-mon[40592]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T08:40:10.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:10.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:10.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:10 smithi043 ceph-mon[29355]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T08:40:10.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:10.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:11 smithi172 ceph-mon[32987]: pgmap v656: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-11T08:40:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:11 smithi172 ceph-mon[32987]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T08:40:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:11 smithi043 ceph-mon[29355]: pgmap v656: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-11T08:40:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:11 smithi043 ceph-mon[29355]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T08:40:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:11 smithi043 ceph-mon[40592]: pgmap v656: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-11T08:40:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:11 smithi043 ceph-mon[40592]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T08:40:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:12 smithi172 ceph-mon[32987]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T08:40:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:12 smithi043 ceph-mon[29355]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T08:40:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:12 smithi043 ceph-mon[40592]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T08:40:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:13 smithi172 ceph-mon[32987]: pgmap v659: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-11T08:40:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:13 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:13 smithi172 ceph-mon[32987]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T08:40:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1b"}]: dispatch 2024-09-11T08:40:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:13 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1b"}]: dispatch 2024-09-11T08:40:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:13 smithi172 ceph-mon[32987]: 223.1b deep-scrub starts 2024-09-11T08:40:13.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:13 smithi172 ceph-mon[32987]: 223.1b deep-scrub ok 2024-09-11T08:40:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[40592]: pgmap v659: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-11T08:40:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[40592]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T08:40:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1b"}]: dispatch 2024-09-11T08:40:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1b"}]: dispatch 2024-09-11T08:40:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[40592]: 223.1b deep-scrub starts 2024-09-11T08:40:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[40592]: 223.1b deep-scrub ok 2024-09-11T08:40:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[29355]: pgmap v659: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-11T08:40:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[29355]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T08:40:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1b"}]: dispatch 2024-09-11T08:40:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1b"}]: dispatch 2024-09-11T08:40:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[29355]: 223.1b deep-scrub starts 2024-09-11T08:40:13.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-mon[29355]: 223.1b deep-scrub ok 2024-09-11T08:40:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:40:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:13] "GET /metrics HTTP/1.1" 200 228646 "" "Prometheus/2.43.0" 2024-09-11T08:40:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:15 smithi172 ceph-mon[32987]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 1.1 KiB/s wr, 154 op/s 2024-09-11T08:40:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:40:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:15 smithi043 ceph-mon[29355]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 1.1 KiB/s wr, 154 op/s 2024-09-11T08:40:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:15 smithi043 ceph-mon[40592]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 1.1 KiB/s wr, 154 op/s 2024-09-11T08:40:16.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:40:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:40:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:17 smithi043 ceph-mon[40592]: pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 853 B/s wr, 121 op/s 2024-09-11T08:40:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:17 smithi043 ceph-mon[29355]: pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 853 B/s wr, 121 op/s 2024-09-11T08:40:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:17.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:17 smithi172 ceph-mon[32987]: pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 853 B/s wr, 121 op/s 2024-09-11T08:40:19.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:19 smithi043 ceph-mon[29355]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 664 B/s wr, 94 op/s 2024-09-11T08:40:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:19 smithi043 ceph-mon[40592]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 664 B/s wr, 94 op/s 2024-09-11T08:40:19.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:19 smithi172 ceph-mon[32987]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 664 B/s wr, 94 op/s 2024-09-11T08:40:21.783 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:21 smithi172 ceph-mon[32987]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 50 KiB/s rd, 587 B/s wr, 84 op/s 2024-09-11T08:40:21.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:21 smithi043 ceph-mon[29355]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 50 KiB/s rd, 587 B/s wr, 84 op/s 2024-09-11T08:40:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:21 smithi043 ceph-mon[40592]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 50 KiB/s rd, 587 B/s wr, 84 op/s 2024-09-11T08:40:23.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:23] "GET /metrics HTTP/1.1" 200 228646 "" "Prometheus/2.43.0" 2024-09-11T08:40:23.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[40592]: pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 511 B/s wr, 73 op/s 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[29355]: pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 511 B/s wr, 73 op/s 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:23.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:23 smithi172 ceph-mon[32987]: pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 511 B/s wr, 73 op/s 2024-09-11T08:40:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:23.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:25 smithi172 ceph-mon[32987]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 437 B/s wr, 63 op/s 2024-09-11T08:40:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:40:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:25 smithi043 ceph-mon[40592]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 437 B/s wr, 63 op/s 2024-09-11T08:40:25.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:25 smithi043 ceph-mon[29355]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 437 B/s wr, 63 op/s 2024-09-11T08:40:26.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:40:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:40:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:27 smithi043 ceph-mon[40592]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:40:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:27 smithi043 ceph-mon[29355]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:40:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:27 smithi172 ceph-mon[32987]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:40:27.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:29 smithi043 ceph-mon[40592]: pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:29 smithi043 ceph-mon[29355]: pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:29 smithi172 ceph-mon[32987]: pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:31 smithi043 ceph-mon[40592]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:31 smithi043 ceph-mon[29355]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:31 smithi172 ceph-mon[32987]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:32.894 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:40:32 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:40:32.508+0000 7f5abb420640 -1 snap_mapper.add_oid found existing snaps mapped on 223:d826c1f5:test-rados-api-smithi043-79312-80::foo:2, removing 2024-09-11T08:40:32.894 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:40:32 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:40:32.508+0000 7f0f5c27d640 -1 snap_mapper.add_oid found existing snaps mapped on 223:d826c1f5:test-rados-api-smithi043-79312-80::foo:2, removing 2024-09-11T08:40:32.981 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:40:32 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:40:32.507+0000 7f6b20860640 -1 snap_mapper.add_oid found existing snaps mapped on 223:d826c1f5:test-rados-api-smithi043-79312-80::foo:2, removing 2024-09-11T08:40:33.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:33] "GET /metrics HTTP/1.1" 200 228642 "" "Prometheus/2.43.0" 2024-09-11T08:40:33.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[29355]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:40:33.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:33.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:33.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-79"}]: dispatch 2024-09-11T08:40:33.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-79"}]: dispatch 2024-09-11T08:40:33.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[40592]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:40:33.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:33.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:33.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-79"}]: dispatch 2024-09-11T08:40:33.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-79"}]: dispatch 2024-09-11T08:40:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:33 smithi172 ceph-mon[32987]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:40:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-79"}]: dispatch 2024-09-11T08:40:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-79"}]: dispatch 2024-09-11T08:40:34.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:34 smithi043 ceph-mon[40592]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T08:40:34.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:34 smithi043 ceph-mon[29355]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T08:40:34.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:34 smithi172 ceph-mon[32987]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T08:40:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:40:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:35 smithi172 ceph-mon[32987]: pgmap v672: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:40:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:35 smithi172 ceph-mon[32987]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T08:40:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:35.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:35.795 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:35 smithi043 ceph-mon[40592]: pgmap v672: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:40:35.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:35 smithi043 ceph-mon[40592]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T08:40:35.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:35.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:35.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:35 smithi043 ceph-mon[29355]: pgmap v672: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:40:35.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:35 smithi043 ceph-mon[29355]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T08:40:35.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:35.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:36.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:40:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:40:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[40592]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T08:40:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:36.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[29355]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T08:40:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:36.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:36 smithi172 ceph-mon[32987]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T08:40:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:36.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[40592]: pgmap v675: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:40:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[40592]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T08:40:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[29355]: pgmap v675: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:40:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[29355]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T08:40:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:37 smithi172 ceph-mon[32987]: pgmap v675: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:40:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:37 smithi172 ceph-mon[32987]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T08:40:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:38 smithi043 ceph-mon[40592]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T08:40:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:38 smithi043 ceph-mon[29355]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T08:40:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:38.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:38 smithi172 ceph-mon[32987]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T08:40:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[40592]: pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[40592]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T08:40:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[29355]: pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[29355]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T08:40:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:39 smithi172 ceph-mon[32987]: pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:39 smithi172 ceph-mon[32987]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T08:40:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:40 smithi043 ceph-mon[40592]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T08:40:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:40:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:40:40.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:40 smithi043 ceph-mon[29355]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T08:40:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:40:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:40:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:40 smithi172 ceph-mon[32987]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T08:40:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:40:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:40:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[40592]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-11T08:40:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[40592]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T08:40:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:40:41.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:40:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[29355]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-11T08:40:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[29355]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T08:40:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:40:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:40:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:41 smithi172 ceph-mon[32987]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-11T08:40:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:41 smithi172 ceph-mon[32987]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T08:40:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:40:41.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:40:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-11T08:40:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[40592]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T08:40:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-11T08:40:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[29355]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T08:40:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-11T08:40:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:42 smithi172 ceph-mon[32987]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T08:40:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:43.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:43] "GET /metrics HTTP/1.1" 200 228620 "" "Prometheus/2.43.0" 2024-09-11T08:40:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T08:40:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-81"}]: dispatch 2024-09-11T08:40:43.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-81"}]: dispatch 2024-09-11T08:40:43.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:43.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:43.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T08:40:43.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:43.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:43.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-81"}]: dispatch 2024-09-11T08:40:43.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-81"}]: dispatch 2024-09-11T08:40:43.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T08:40:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-81"}]: dispatch 2024-09-11T08:40:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-81"}]: dispatch 2024-09-11T08:40:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]: dispatch 2024-09-11T08:40:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]: dispatch 2024-09-11T08:40:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:44 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]': finished 2024-09-11T08:40:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:44 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]': finished 2024-09-11T08:40:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:44 smithi043 ceph-mon[40592]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T08:40:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:44 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]': finished 2024-09-11T08:40:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:44 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]': finished 2024-09-11T08:40:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:44 smithi043 ceph-mon[29355]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T08:40:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:44 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.8", "id": [2, 6]}]': finished 2024-09-11T08:40:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:44 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "224.1b", "id": [2, 6]}]': finished 2024-09-11T08:40:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:44 smithi172 ceph-mon[32987]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T08:40:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:45 smithi172 ceph-mon[32987]: pgmap v687: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-11T08:40:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:45 smithi172 ceph-mon[32987]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T08:40:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:45.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:40:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:45 smithi043 ceph-mon[40592]: pgmap v687: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-11T08:40:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:45 smithi043 ceph-mon[40592]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T08:40:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:45 smithi043 ceph-mon[29355]: pgmap v687: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-11T08:40:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:45 smithi043 ceph-mon[29355]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T08:40:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:46.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:40:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:40:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[40592]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T08:40:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[29355]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T08:40:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:46.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:46 smithi172 ceph-mon[32987]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T08:40:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:46.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[40592]: pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-11T08:40:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[40592]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T08:40:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[29355]: pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-11T08:40:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[29355]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T08:40:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:47 smithi172 ceph-mon[32987]: pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-11T08:40:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:47 smithi172 ceph-mon[32987]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T08:40:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[40592]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T08:40:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[40592]: pgmap v693: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[29355]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T08:40:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:48 smithi043 ceph-mon[29355]: pgmap v693: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:48 smithi172 ceph-mon[32987]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T08:40:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:48 smithi172 ceph-mon[32987]: pgmap v693: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:49.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:49.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:49 smithi043 ceph-mon[29355]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T08:40:49.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:49.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:49 smithi043 ceph-mon[40592]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T08:40:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:49.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:49 smithi172 ceph-mon[32987]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T08:40:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:50.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:50.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:50 smithi043 ceph-mon[29355]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T08:40:50.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:50 smithi043 ceph-mon[29355]: pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:50.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:50.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:50 smithi043 ceph-mon[40592]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T08:40:50.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:50 smithi043 ceph-mon[40592]: pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:50 smithi172 ceph-mon[32987]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T08:40:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:50 smithi172 ceph-mon[32987]: pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:51 smithi043 ceph-mon[29355]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T08:40:51.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:51 smithi043 ceph-mon[40592]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T08:40:51.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:51 smithi172 ceph-mon[32987]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T08:40:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:52 smithi172 ceph-mon[32987]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T08:40:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-83"}]: dispatch 2024-09-11T08:40:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-83"}]: dispatch 2024-09-11T08:40:52.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:52 smithi172 ceph-mon[32987]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:53.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[40592]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T08:40:53.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-83"}]: dispatch 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-83"}]: dispatch 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[40592]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[29355]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-83"}]: dispatch 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-83"}]: dispatch 2024-09-11T08:40:53.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:52 smithi043 ceph-mon[29355]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:53 smithi172 ceph-mon[32987]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T08:40:54.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:53 smithi043 ceph-mon[29355]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T08:40:54.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:53 smithi043 ceph-mon[40592]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T08:40:54.145 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:40:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:53] "GET /metrics HTTP/1.1" 200 228620 "" "Prometheus/2.43.0" 2024-09-11T08:40:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:54 smithi172 ceph-mon[32987]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T08:40:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:54 smithi172 ceph-mon[32987]: pgmap v702: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-11T08:40:55.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:54 smithi043 ceph-mon[40592]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T08:40:55.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:55.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:55.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:54 smithi043 ceph-mon[40592]: pgmap v702: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-11T08:40:55.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:54 smithi043 ceph-mon[29355]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T08:40:55.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:55.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:40:55.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:54 smithi043 ceph-mon[29355]: pgmap v702: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-11T08:40:55.663 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:40:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:40:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:55.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:55 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:55.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:55.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:55 smithi172 ceph-mon[32987]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T08:40:55.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:55.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:55.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:56.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:56.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:56.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[40592]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T08:40:56.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:56.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:56.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:40:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:40:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[29355]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T08:40:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:40:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:56.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:40:56.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:40:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:40:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:56 smithi172 ceph-mon[32987]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T08:40:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:56.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:56.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:56 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:56.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:56 smithi172 ceph-mon[32987]: pgmap v705: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-11T08:40:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[40592]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T08:40:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:57.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[40592]: pgmap v705: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-11T08:40:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:40:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[29355]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T08:40:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:40:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:57.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:56 smithi043 ceph-mon[29355]: pgmap v705: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:57 smithi172 ceph-mon[32987]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[40592]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T08:40:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:58.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:58.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_tier","val": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:40:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[29355]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T08:40:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:40:58.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:58 smithi172 ceph-mon[32987]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T08:40:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:58 smithi172 ceph-mon[32987]: pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[40592]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T08:40:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:59.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[40592]: pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:40:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[29355]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T08:40:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:40:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:58 smithi043 ceph-mon[29355]: pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:40:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:40:59 smithi172 ceph-mon[32987]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T08:41:00.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:41:00.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:40:59 smithi043 ceph-mon[40592]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T08:41:00.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:41:00.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:40:59 smithi043 ceph-mon[29355]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T08:41:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:00 smithi172 ceph-mon[32987]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T08:41:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:00 smithi172 ceph-mon[32987]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:01.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:00 smithi043 ceph-mon[40592]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T08:41:01.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:00 smithi043 ceph-mon[40592]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:01.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:00 smithi043 ceph-mon[29355]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T08:41:01.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:00 smithi043 ceph-mon[29355]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:01.700 INFO:tasks.workunit.client.0.smithi043.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18518 ms) 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (21967 ms) 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3025 ms) 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3017 ms) 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-11T08:41:01.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3018 ms) 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3022 ms) 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4027 ms) 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38873 ms) 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16987 ms) 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-11T08:41:01.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4031 ms) 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3020 ms) 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25073 ms) 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15045 ms) 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5044 ms) 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4024 ms) 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrubs... 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25055 ms) 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10079 ms) 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-11T08:41:01.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9068 ms) 2024-09-11T08:41:01.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-11T08:41:01.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:41:01.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9045 ms) 2024-09-11T08:41:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:01 smithi172 ceph-mon[32987]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T08:41:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-85"}]: dispatch 2024-09-11T08:41:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-85"}]: dispatch 2024-09-11T08:41:02.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[29355]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T08:41:02.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:02.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:02.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-85"}]: dispatch 2024-09-11T08:41:02.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-85"}]: dispatch 2024-09-11T08:41:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[40592]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T08:41:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-85"}]: dispatch 2024-09-11T08:41:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-85"}]: dispatch 2024-09-11T08:41:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:02 smithi172 ceph-mon[32987]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T08:41:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:02 smithi172 ceph-mon[32987]: pgmap v714: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:02 smithi043 ceph-mon[40592]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T08:41:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:02 smithi043 ceph-mon[40592]: pgmap v714: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:02 smithi043 ceph-mon[29355]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T08:41:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:02 smithi043 ceph-mon[29355]: pgmap v714: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:03 smithi172 ceph-mon[32987]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T08:41:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:03 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:03] "GET /metrics HTTP/1.1" 200 228694 "" "Prometheus/2.43.0" 2024-09-11T08:41:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-mon[40592]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T08:41:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-mon[29355]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T08:41:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:03 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:04 smithi172 ceph-mon[32987]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T08:41:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:04 smithi172 ceph-mon[32987]: pgmap v717: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:41:05.045 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:04 smithi043 ceph-mon[40592]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T08:41:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:04 smithi043 ceph-mon[40592]: pgmap v717: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:41:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:04 smithi043 ceph-mon[29355]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T08:41:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:04 smithi043 ceph-mon[29355]: pgmap v717: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:41:05.726 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:41:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:05 smithi172 ceph-mon[32987]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T08:41:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:05 smithi043 ceph-mon[40592]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T08:41:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:05 smithi043 ceph-mon[29355]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T08:41:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:06.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:41:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:41:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:41:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[40592]: osdmap e466: 8 total, 8 up, 8 in 2024-09-11T08:41:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]: dispatch 2024-09-11T08:41:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]: dispatch 2024-09-11T08:41:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:07.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[40592]: pgmap v720: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:41:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:41:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[29355]: osdmap e466: 8 total, 8 up, 8 in 2024-09-11T08:41:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]: dispatch 2024-09-11T08:41:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]: dispatch 2024-09-11T08:41:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:06 smithi043 ceph-mon[29355]: pgmap v720: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:41:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:41:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:06 smithi172 ceph-mon[32987]: osdmap e466: 8 total, 8 up, 8 in 2024-09-11T08:41:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]: dispatch 2024-09-11T08:41:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]: dispatch 2024-09-11T08:41:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:06 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:07.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:06 smithi172 ceph-mon[32987]: pgmap v720: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:41:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]': finished 2024-09-11T08:41:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:07 smithi043 ceph-mon[40592]: osdmap e467: 8 total, 8 up, 8 in 2024-09-11T08:41:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:41:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:41:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]': finished 2024-09-11T08:41:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:07 smithi043 ceph-mon[29355]: osdmap e467: 8 total, 8 up, 8 in 2024-09-11T08:41:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:41:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:41:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_tier","val": "test-rados-api-smithi043-79312-89-test-flush"}]': finished 2024-09-11T08:41:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:07 smithi172 ceph-mon[32987]: osdmap e467: 8 total, 8 up, 8 in 2024-09-11T08:41:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:41:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:41:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:41:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[40592]: osdmap e468: 8 total, 8 up, 8 in 2024-09-11T08:41:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:41:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:41:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[40592]: pgmap v723: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[40592]: osdmap e469: 8 total, 8 up, 8 in 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[29355]: osdmap e468: 8 total, 8 up, 8 in 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[29355]: pgmap v723: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:41:09.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:08 smithi043 ceph-mon[29355]: osdmap e469: 8 total, 8 up, 8 in 2024-09-11T08:41:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:41:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:08 smithi172 ceph-mon[32987]: osdmap e468: 8 total, 8 up, 8 in 2024-09-11T08:41:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:41:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:41:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:08 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:08 smithi172 ceph-mon[32987]: pgmap v723: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:41:09.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:08 smithi172 ceph-mon[32987]: osdmap e469: 8 total, 8 up, 8 in 2024-09-11T08:41:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[40592]: osdmap e470: 8 total, 8 up, 8 in 2024-09-11T08:41:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-87"}]: dispatch 2024-09-11T08:41:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-87"}]: dispatch 2024-09-11T08:41:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[40592]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[29355]: osdmap e470: 8 total, 8 up, 8 in 2024-09-11T08:41:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-87"}]: dispatch 2024-09-11T08:41:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-87"}]: dispatch 2024-09-11T08:41:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:10 smithi043 ceph-mon[29355]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:11.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:10 smithi172 ceph-mon[32987]: osdmap e470: 8 total, 8 up, 8 in 2024-09-11T08:41:11.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:11.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:11.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-87"}]: dispatch 2024-09-11T08:41:11.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-87"}]: dispatch 2024-09-11T08:41:11.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:10 smithi172 ceph-mon[32987]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:12.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:11 smithi043 ceph-mon[29355]: osdmap e471: 8 total, 8 up, 8 in 2024-09-11T08:41:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:11 smithi043 ceph-mon[40592]: osdmap e471: 8 total, 8 up, 8 in 2024-09-11T08:41:12.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:11 smithi172 ceph-mon[32987]: osdmap e471: 8 total, 8 up, 8 in 2024-09-11T08:41:13.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[29355]: osdmap e472: 8 total, 8 up, 8 in 2024-09-11T08:41:13.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:13.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:13.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[29355]: pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:13.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[40592]: osdmap e472: 8 total, 8 up, 8 in 2024-09-11T08:41:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[40592]: pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:12 smithi172 ceph-mon[32987]: osdmap e472: 8 total, 8 up, 8 in 2024-09-11T08:41:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:12 smithi172 ceph-mon[32987]: pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:13] "GET /metrics HTTP/1.1" 200 228708 "" "Prometheus/2.43.0" 2024-09-11T08:41:14.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[29355]: osdmap e473: 8 total, 8 up, 8 in 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[40592]: osdmap e473: 8 total, 8 up, 8 in 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:13 smithi172 ceph-mon[32987]: osdmap e473: 8 total, 8 up, 8 in 2024-09-11T08:41:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:41:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:41:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:14 smithi043 ceph-mon[40592]: osdmap e474: 8 total, 8 up, 8 in 2024-09-11T08:41:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:14 smithi043 ceph-mon[40592]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:41:15.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:41:15.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:14 smithi043 ceph-mon[29355]: osdmap e474: 8 total, 8 up, 8 in 2024-09-11T08:41:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:14 smithi043 ceph-mon[29355]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:41:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:41:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:14 smithi172 ceph-mon[32987]: osdmap e474: 8 total, 8 up, 8 in 2024-09-11T08:41:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:14 smithi172 ceph-mon[32987]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:41:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:41:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:15 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:15 smithi043 ceph-mon[40592]: osdmap e475: 8 total, 8 up, 8 in 2024-09-11T08:41:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:15 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:15 smithi043 ceph-mon[29355]: osdmap e475: 8 total, 8 up, 8 in 2024-09-11T08:41:16.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:41:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:41:16.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:15 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:16.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:15 smithi172 ceph-mon[32987]: osdmap e475: 8 total, 8 up, 8 in 2024-09-11T08:41:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[40592]: osdmap e476: 8 total, 8 up, 8 in 2024-09-11T08:41:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-90"}]: dispatch 2024-09-11T08:41:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-90"}]: dispatch 2024-09-11T08:41:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[40592]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:41:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[29355]: osdmap e476: 8 total, 8 up, 8 in 2024-09-11T08:41:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-90"}]: dispatch 2024-09-11T08:41:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-90"}]: dispatch 2024-09-11T08:41:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:16 smithi043 ceph-mon[29355]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:41:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:16 smithi172 ceph-mon[32987]: osdmap e476: 8 total, 8 up, 8 in 2024-09-11T08:41:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-90"}]: dispatch 2024-09-11T08:41:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-90"}]: dispatch 2024-09-11T08:41:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:16 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:16 smithi172 ceph-mon[32987]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:41:18.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:17 smithi043 ceph-mon[40592]: osdmap e477: 8 total, 8 up, 8 in 2024-09-11T08:41:18.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:17 smithi043 ceph-mon[29355]: osdmap e477: 8 total, 8 up, 8 in 2024-09-11T08:41:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:17 smithi172 ceph-mon[32987]: osdmap e477: 8 total, 8 up, 8 in 2024-09-11T08:41:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[29355]: osdmap e478: 8 total, 8 up, 8 in 2024-09-11T08:41:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[29355]: pgmap v738: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:19.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[29355]: osdmap e479: 8 total, 8 up, 8 in 2024-09-11T08:41:19.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[40592]: osdmap e478: 8 total, 8 up, 8 in 2024-09-11T08:41:19.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:19.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:19.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[40592]: pgmap v738: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:19.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:19.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:18 smithi043 ceph-mon[40592]: osdmap e479: 8 total, 8 up, 8 in 2024-09-11T08:41:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:18 smithi172 ceph-mon[32987]: osdmap e478: 8 total, 8 up, 8 in 2024-09-11T08:41:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:18 smithi172 ceph-mon[32987]: pgmap v738: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:18 smithi172 ceph-mon[32987]: osdmap e479: 8 total, 8 up, 8 in 2024-09-11T08:41:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:21.132 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:20 smithi172 ceph-mon[32987]: osdmap e480: 8 total, 8 up, 8 in 2024-09-11T08:41:21.132 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:20 smithi172 ceph-mon[32987]: pgmap v741: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:41:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:20 smithi043 ceph-mon[40592]: osdmap e480: 8 total, 8 up, 8 in 2024-09-11T08:41:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:20 smithi043 ceph-mon[40592]: pgmap v741: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:41:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:20 smithi043 ceph-mon[29355]: osdmap e480: 8 total, 8 up, 8 in 2024-09-11T08:41:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:20 smithi043 ceph-mon[29355]: pgmap v741: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:41:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[40592]: osdmap e481: 8 total, 8 up, 8 in 2024-09-11T08:41:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-92"}]: dispatch 2024-09-11T08:41:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-92"}]: dispatch 2024-09-11T08:41:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[29355]: osdmap e481: 8 total, 8 up, 8 in 2024-09-11T08:41:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-92"}]: dispatch 2024-09-11T08:41:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-92"}]: dispatch 2024-09-11T08:41:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:21 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:21 smithi172 ceph-mon[32987]: osdmap e481: 8 total, 8 up, 8 in 2024-09-11T08:41:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-92"}]: dispatch 2024-09-11T08:41:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-92"}]: dispatch 2024-09-11T08:41:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:22 smithi043 ceph-mon[40592]: osdmap e482: 8 total, 8 up, 8 in 2024-09-11T08:41:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:22 smithi043 ceph-mon[40592]: pgmap v744: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:41:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:22 smithi043 ceph-mon[29355]: osdmap e482: 8 total, 8 up, 8 in 2024-09-11T08:41:23.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:22 smithi043 ceph-mon[29355]: pgmap v744: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:41:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:22 smithi172 ceph-mon[32987]: osdmap e482: 8 total, 8 up, 8 in 2024-09-11T08:41:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:22 smithi172 ceph-mon[32987]: pgmap v744: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:41:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:23] "GET /metrics HTTP/1.1" 200 228708 "" "Prometheus/2.43.0" 2024-09-11T08:41:24.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: osdmap e483: 8 total, 8 up, 8 in 2024-09-11T08:41:24.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:24.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:24.146 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: osdmap e483: 8 total, 8 up, 8 in 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:24.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:23 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: osdmap e483: 8 total, 8 up, 8 in 2024-09-11T08:41:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:41:24.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:24.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:24.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:24.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:24.233 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:23 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi172 to 3248M 2024-09-11T08:41:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[40592]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-11T08:41:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[40592]: osdmap e484: 8 total, 8 up, 8 in 2024-09-11T08:41:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[40592]: pgmap v747: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:41:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi172 to 3248M 2024-09-11T08:41:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[29355]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-11T08:41:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[29355]: osdmap e484: 8 total, 8 up, 8 in 2024-09-11T08:41:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:24 smithi043 ceph-mon[29355]: pgmap v747: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:41:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:24 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi172 to 3248M 2024-09-11T08:41:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:24 smithi172 ceph-mon[32987]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-11T08:41:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:24 smithi172 ceph-mon[32987]: osdmap e484: 8 total, 8 up, 8 in 2024-09-11T08:41:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:24 smithi172 ceph-mon[32987]: pgmap v747: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:41:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:41:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:26.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:25 smithi043 ceph-mon[40592]: osdmap e485: 8 total, 8 up, 8 in 2024-09-11T08:41:26.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:25 smithi043 ceph-mon[29355]: osdmap e485: 8 total, 8 up, 8 in 2024-09-11T08:41:26.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:25 smithi172 ceph-mon[32987]: osdmap e485: 8 total, 8 up, 8 in 2024-09-11T08:41:26.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:41:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:41:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:26 smithi043 ceph-mon[40592]: osdmap e486: 8 total, 8 up, 8 in 2024-09-11T08:41:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:26 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:26 smithi043 ceph-mon[40592]: pgmap v750: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:41:27.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:26 smithi043 ceph-mon[29355]: osdmap e486: 8 total, 8 up, 8 in 2024-09-11T08:41:27.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:26 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:27.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:26 smithi043 ceph-mon[29355]: pgmap v750: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:41:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:26 smithi172 ceph-mon[32987]: osdmap e486: 8 total, 8 up, 8 in 2024-09-11T08:41:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:26 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:26 smithi172 ceph-mon[32987]: pgmap v750: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:41:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:28.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:28 smithi043 ceph-mon[40592]: pgmap v751: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 573 B/s wr, 1 op/s 2024-09-11T08:41:29.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:28 smithi043 ceph-mon[29355]: pgmap v751: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 573 B/s wr, 1 op/s 2024-09-11T08:41:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:28 smithi172 ceph-mon[32987]: pgmap v751: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 573 B/s wr, 1 op/s 2024-09-11T08:41:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:31 smithi043 ceph-mon[40592]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:41:31.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:31 smithi043 ceph-mon[29355]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:41:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:31 smithi172 ceph-mon[32987]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:41:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:33 smithi043 ceph-mon[40592]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:41:33.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:33 smithi043 ceph-mon[29355]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:41:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:33 smithi172 ceph-mon[32987]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:41:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:41:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:33] "GET /metrics HTTP/1.1" 200 228697 "" "Prometheus/2.43.0" 2024-09-11T08:41:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:35 smithi043 ceph-mon[40592]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 876 B/s wr, 3 op/s 2024-09-11T08:41:35.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:35 smithi043 ceph-mon[29355]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 876 B/s wr, 3 op/s 2024-09-11T08:41:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:35 smithi172 ceph-mon[32987]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 876 B/s wr, 3 op/s 2024-09-11T08:41:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:41:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:36.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:41:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:41:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:37 smithi043 ceph-mon[40592]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 791 B/s wr, 3 op/s 2024-09-11T08:41:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:37 smithi043 ceph-mon[40592]: osdmap e487: 8 total, 8 up, 8 in 2024-09-11T08:41:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:37 smithi043 ceph-mon[29355]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 791 B/s wr, 3 op/s 2024-09-11T08:41:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:37 smithi043 ceph-mon[29355]: osdmap e487: 8 total, 8 up, 8 in 2024-09-11T08:41:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:37 smithi172 ceph-mon[32987]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 791 B/s wr, 3 op/s 2024-09-11T08:41:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:37 smithi172 ceph-mon[32987]: osdmap e487: 8 total, 8 up, 8 in 2024-09-11T08:41:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:39 smithi043 ceph-mon[40592]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-11T08:41:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:39 smithi043 ceph-mon[29355]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-11T08:41:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:39 smithi172 ceph-mon[32987]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-11T08:41:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:41 smithi043 ceph-mon[40592]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:41 smithi043 ceph-mon[29355]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:41 smithi172 ceph-mon[32987]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:42.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:42.481 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:41:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore.transactions t=2024-09-11T08:41:42.1582815Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:41:42.481 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:41:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=cleanup t=2024-09-11T08:41:42.160884779Z level=info msg="Completed cleanup jobs" duration=13.016685ms 2024-09-11T08:41:42.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:42.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:43 smithi043 ceph-mon[40592]: pgmap v759: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:43.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:43 smithi043 ceph-mon[40592]: osdmap e488: 8 total, 8 up, 8 in 2024-09-11T08:41:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:43 smithi043 ceph-mon[29355]: pgmap v759: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:43 smithi043 ceph-mon[29355]: osdmap e488: 8 total, 8 up, 8 in 2024-09-11T08:41:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:43 smithi172 ceph-mon[32987]: pgmap v759: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:43 smithi172 ceph-mon[32987]: osdmap e488: 8 total, 8 up, 8 in 2024-09-11T08:41:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:41:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:43] "GET /metrics HTTP/1.1" 200 228638 "" "Prometheus/2.43.0" 2024-09-11T08:41:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:45 smithi043 ceph-mon[40592]: pgmap v761: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:45 smithi043 ceph-mon[29355]: pgmap v761: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:45 smithi172 ceph-mon[32987]: pgmap v761: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:41:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:46.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:41:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:41:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[29355]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-94"}]: dispatch 2024-09-11T08:41:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-94"}]: dispatch 2024-09-11T08:41:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[40592]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-94"}]: dispatch 2024-09-11T08:41:47.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-94"}]: dispatch 2024-09-11T08:41:47.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:47.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:47 smithi172 ceph-mon[32987]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:41:47.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:47.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:41:47.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-94"}]: dispatch 2024-09-11T08:41:47.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-94"}]: dispatch 2024-09-11T08:41:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:48 smithi043 ceph-mon[40592]: osdmap e489: 8 total, 8 up, 8 in 2024-09-11T08:41:48.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:48 smithi043 ceph-mon[29355]: osdmap e489: 8 total, 8 up, 8 in 2024-09-11T08:41:48.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:48 smithi172 ceph-mon[32987]: osdmap e489: 8 total, 8 up, 8 in 2024-09-11T08:41:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:49 smithi043 ceph-mon[40592]: pgmap v764: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:49 smithi043 ceph-mon[40592]: osdmap e490: 8 total, 8 up, 8 in 2024-09-11T08:41:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:49 smithi043 ceph-mon[29355]: pgmap v764: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:49.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:49 smithi043 ceph-mon[29355]: osdmap e490: 8 total, 8 up, 8 in 2024-09-11T08:41:49.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:49.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:49 smithi172 ceph-mon[32987]: pgmap v764: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:49 smithi172 ceph-mon[32987]: osdmap e490: 8 total, 8 up, 8 in 2024-09-11T08:41:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:50 smithi043 ceph-mon[40592]: osdmap e491: 8 total, 8 up, 8 in 2024-09-11T08:41:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:50.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:50.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:50 smithi043 ceph-mon[29355]: osdmap e491: 8 total, 8 up, 8 in 2024-09-11T08:41:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:50 smithi172 ceph-mon[32987]: osdmap e491: 8 total, 8 up, 8 in 2024-09-11T08:41:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:41:51.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:51 smithi172 ceph-mon[32987]: pgmap v767: 228 pgs: 5 creating+peering, 27 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:51.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:51 smithi172 ceph-mon[32987]: osdmap e492: 8 total, 8 up, 8 in 2024-09-11T08:41:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:51 smithi043 ceph-mon[40592]: pgmap v767: 228 pgs: 5 creating+peering, 27 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:51 smithi043 ceph-mon[40592]: osdmap e492: 8 total, 8 up, 8 in 2024-09-11T08:41:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:51 smithi043 ceph-mon[29355]: pgmap v767: 228 pgs: 5 creating+peering, 27 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:41:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:51 smithi043 ceph-mon[29355]: osdmap e492: 8 total, 8 up, 8 in 2024-09-11T08:41:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:53 smithi043 ceph-mon[40592]: pgmap v769: 228 pgs: 5 creating+peering, 27 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 208 B/s wr, 1 op/s 2024-09-11T08:41:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:53 smithi043 ceph-mon[29355]: pgmap v769: 228 pgs: 5 creating+peering, 27 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 208 B/s wr, 1 op/s 2024-09-11T08:41:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:53 smithi172 ceph-mon[32987]: pgmap v769: 228 pgs: 5 creating+peering, 27 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 208 B/s wr, 1 op/s 2024-09-11T08:41:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:41:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:53] "GET /metrics HTTP/1.1" 200 228638 "" "Prometheus/2.43.0" 2024-09-11T08:41:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:55 smithi043 ceph-mon[40592]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:41:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:55 smithi043 ceph-mon[29355]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:41:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:55 smithi172 ceph-mon[32987]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:41:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:41:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:41:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:56.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:41:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:41:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:57 smithi043 ceph-mon[40592]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 647 B/s rd, 647 B/s wr, 2 op/s 2024-09-11T08:41:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:57 smithi043 ceph-mon[29355]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 647 B/s rd, 647 B/s wr, 2 op/s 2024-09-11T08:41:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:57 smithi172 ceph-mon[32987]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 647 B/s rd, 647 B/s wr, 2 op/s 2024-09-11T08:41:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:41:59 smithi043 ceph-mon[40592]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 574 B/s rd, 574 B/s wr, 1 op/s 2024-09-11T08:41:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:41:59 smithi043 ceph-mon[29355]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 574 B/s rd, 574 B/s wr, 1 op/s 2024-09-11T08:41:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:41:59 smithi172 ceph-mon[32987]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 574 B/s rd, 574 B/s wr, 1 op/s 2024-09-11T08:42:00.644 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:42:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:42:00.336+0000 7f5ab941c640 -1 snap_mapper.add_oid found existing snaps mapped on 63:f3b32f51:test-rados-api-smithi043-79312-97::foo:31, removing 2024-09-11T08:42:00.645 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:42:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2[56903]: 2024-09-11T08:42:00.336+0000 7f5ce7aa4640 -1 snap_mapper.add_oid found existing snaps mapped on 63:f3b32f51:test-rados-api-smithi043-79312-97::foo:31, removing 2024-09-11T08:42:00.645 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:42:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:42:00.336+0000 7f0f60285640 -1 snap_mapper.add_oid found existing snaps mapped on 63:f3b32f51:test-rados-api-smithi043-79312-97::foo:31, removing 2024-09-11T08:42:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[29355]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:42:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-96"}]: dispatch 2024-09-11T08:42:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-96"}]: dispatch 2024-09-11T08:42:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[40592]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:42:01.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:01.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:01.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-96"}]: dispatch 2024-09-11T08:42:01.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-96"}]: dispatch 2024-09-11T08:42:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:01 smithi172 ceph-mon[32987]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:42:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-96"}]: dispatch 2024-09-11T08:42:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-96"}]: dispatch 2024-09-11T08:42:02.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:02 smithi043 ceph-mon[40592]: osdmap e493: 8 total, 8 up, 8 in 2024-09-11T08:42:02.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:02 smithi043 ceph-mon[29355]: osdmap e493: 8 total, 8 up, 8 in 2024-09-11T08:42:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:02 smithi172 ceph-mon[32987]: osdmap e493: 8 total, 8 up, 8 in 2024-09-11T08:42:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-mon[40592]: pgmap v775: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-11T08:42:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-mon[40592]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T08:42:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:03.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-mon[29355]: pgmap v775: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-11T08:42:03.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-mon[29355]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T08:42:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:03 smithi172 ceph-mon[32987]: pgmap v775: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-11T08:42:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:03 smithi172 ceph-mon[32987]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T08:42:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:42:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:03] "GET /metrics HTTP/1.1" 200 228663 "" "Prometheus/2.43.0" 2024-09-11T08:42:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:04 smithi043 ceph-mon[40592]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T08:42:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:04.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:04.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:04.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:04 smithi043 ceph-mon[29355]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T08:42:04.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:04.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:04 smithi172 ceph-mon[32987]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T08:42:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:05.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[29355]: pgmap v778: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-11T08:42:05.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:42:05.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[29355]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T08:42:05.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]: dispatch 2024-09-11T08:42:05.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]: dispatch 2024-09-11T08:42:05.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[40592]: pgmap v778: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-11T08:42:05.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:42:05.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[40592]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T08:42:05.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]: dispatch 2024-09-11T08:42:05.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]: dispatch 2024-09-11T08:42:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:42:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:05 smithi172 ceph-mon[32987]: pgmap v778: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-11T08:42:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:42:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:05 smithi172 ceph-mon[32987]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T08:42:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]: dispatch 2024-09-11T08:42:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]: dispatch 2024-09-11T08:42:06.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:42:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:42:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]': finished 2024-09-11T08:42:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[40592]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T08:42:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]': finished 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[40592]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]': finished 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[29355]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]': finished 2024-09-11T08:42:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:06 smithi043 ceph-mon[29355]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T08:42:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:06 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-98", "mode": "writeback"}]': finished 2024-09-11T08:42:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:06 smithi172 ceph-mon[32987]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T08:42:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-98"}]': finished 2024-09-11T08:42:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:06 smithi172 ceph-mon[32987]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T08:42:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: pgmap v781: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: pgmap v781: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:42:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:42:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T08:42:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:42:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:42:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: pgmap v781: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:42:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:42:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:42:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T08:42:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:42:07.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:42:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[40592]: pgmap v784: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]': finished 2024-09-11T08:42:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[40592]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T08:42:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[29355]: pgmap v784: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]': finished 2024-09-11T08:42:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[29355]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T08:42:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:09 smithi172 ceph-mon[32987]: pgmap v784: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_count","val": "1"}]': finished 2024-09-11T08:42:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:09 smithi172 ceph-mon[32987]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T08:42:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:09.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:10.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:10.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:10 smithi043 ceph-mon[40592]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T08:42:10.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:42:10.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:42:10.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:10.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:10 smithi043 ceph-mon[29355]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T08:42:10.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:42:10.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:42:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:10 smithi172 ceph-mon[32987]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T08:42:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:42:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:42:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[40592]: pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]': finished 2024-09-11T08:42:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[40592]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T08:42:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[29355]: pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]': finished 2024-09-11T08:42:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[29355]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T08:42:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:11 smithi172 ceph-mon[32987]: pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-98","var": "target_max_objects","val": "250"}]': finished 2024-09-11T08:42:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:11 smithi172 ceph-mon[32987]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T08:42:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:42:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[40592]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T08:42:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]': finished 2024-09-11T08:42:12.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[40592]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T08:42:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:42:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[29355]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T08:42:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]': finished 2024-09-11T08:42:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:12 smithi043 ceph-mon[29355]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T08:42:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:42:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:12 smithi172 ceph-mon[32987]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T08:42:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]: dispatch 2024-09-11T08:42:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-98"}]': finished 2024-09-11T08:42:12.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:12 smithi172 ceph-mon[32987]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T08:42:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:13 smithi172 ceph-mon[32987]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:13 smithi043 ceph-mon[40592]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:13 smithi043 ceph-mon[29355]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:42:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:42:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:13] "GET /metrics HTTP/1.1" 200 228656 "" "Prometheus/2.43.0" 2024-09-11T08:42:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:14 smithi172 ceph-mon[32987]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T08:42:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:14 smithi043 ceph-mon[40592]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T08:42:14.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:14 smithi043 ceph-mon[29355]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T08:42:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:42:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:15 smithi172 ceph-mon[32987]: pgmap v793: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:15 smithi172 ceph-mon[32987]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T08:42:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:15 smithi043 ceph-mon[40592]: pgmap v793: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:15 smithi043 ceph-mon[40592]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T08:42:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:15 smithi043 ceph-mon[29355]: pgmap v793: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:15 smithi043 ceph-mon[29355]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T08:42:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:16.423 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:42:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:42:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:16 smithi172 ceph-mon[32987]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T08:42:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:16.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:16 smithi043 ceph-mon[40592]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T08:42:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:16.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:16 smithi043 ceph-mon[29355]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T08:42:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:16.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:42:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:17 smithi172 ceph-mon[32987]: pgmap v796: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:42:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:17 smithi172 ceph-mon[32987]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T08:42:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[40592]: pgmap v796: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:42:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[40592]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T08:42:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[29355]: pgmap v796: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:42:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[29355]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T08:42:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]': finished 2024-09-11T08:42:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:18 smithi172 ceph-mon[32987]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T08:42:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]: dispatch 2024-09-11T08:42:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]: dispatch 2024-09-11T08:42:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:18 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]': finished 2024-09-11T08:42:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[40592]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T08:42:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]: dispatch 2024-09-11T08:42:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]: dispatch 2024-09-11T08:42:18.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-6", "overlaypool": "test-rados-api-smithi043-79312-100"}]': finished 2024-09-11T08:42:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[29355]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T08:42:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]: dispatch 2024-09-11T08:42:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]: dispatch 2024-09-11T08:42:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:18 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:19 smithi172 ceph-mon[32987]: pgmap v799: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]': finished 2024-09-11T08:42:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:19 smithi172 ceph-mon[32987]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T08:42:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[40592]: pgmap v799: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]': finished 2024-09-11T08:42:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[40592]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T08:42:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[29355]: pgmap v799: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-100", "mode": "writeback"}]': finished 2024-09-11T08:42:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[29355]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T08:42:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:42:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:20 smithi172 ceph-mon[32987]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T08:42:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:20.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:42:20.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:20 smithi043 ceph-mon[40592]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T08:42:20.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:20.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:42:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:20 smithi043 ceph-mon[29355]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T08:42:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:21 smithi172 ceph-mon[32987]: pgmap v802: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:21 smithi172 ceph-mon[32987]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T08:42:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:21 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[40592]: pgmap v802: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[40592]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T08:42:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[29355]: pgmap v802: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[29355]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T08:42:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:21 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:42:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:22 smithi172 ceph-mon[32987]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T08:42:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:42:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:42:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:42:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:22 smithi043 ceph-mon[40592]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T08:42:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:42:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:42:22.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:42:22.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:22 smithi043 ceph-mon[29355]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T08:42:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:42:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:42:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:23 smithi172 ceph-mon[32987]: pgmap v805: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-11T08:42:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:23 smithi172 ceph-mon[32987]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T08:42:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[29355]: pgmap v805: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-11T08:42:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[29355]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T08:42:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[40592]: pgmap v805: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-11T08:42:23.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[40592]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T08:42:23.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:23.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:42:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:23] "GET /metrics HTTP/1.1" 200 228656 "" "Prometheus/2.43.0" 2024-09-11T08:42:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:42:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:24 smithi172 ceph-mon[32987]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T08:42:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:24 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:42:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[40592]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T08:42:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]': finished 2024-09-11T08:42:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[29355]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T08:42:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]: dispatch 2024-09-11T08:42:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:24 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:25.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:25 smithi043 ceph-mon[40592]: pgmap v808: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:42:25.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]': finished 2024-09-11T08:42:25.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:25 smithi043 ceph-mon[40592]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T08:42:25.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:25 smithi043 ceph-mon[29355]: pgmap v808: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:42:25.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]': finished 2024-09-11T08:42:25.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:25 smithi043 ceph-mon[29355]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T08:42:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:25 smithi172 ceph-mon[32987]: pgmap v808: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:42:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-100"}]': finished 2024-09-11T08:42:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:25 smithi172 ceph-mon[32987]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T08:42:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:42:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:26.475 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:42:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:42:26.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:26 smithi043 ceph-mon[40592]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T08:42:26.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:26 smithi043 ceph-mon[29355]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T08:42:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:26 smithi172 ceph-mon[32987]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T08:42:27.810 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:27.810 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:27 smithi172 ceph-mon[32987]: pgmap v811: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:27.810 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:27 smithi172 ceph-mon[32987]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T08:42:27.811 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:27.811 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:27.811 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:27 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:27.811 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:27 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:27.811 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[40592]: pgmap v811: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[40592]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T08:42:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:27.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:27.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[29355]: pgmap v811: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:27.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[29355]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T08:42:27.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:27.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:27.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:27.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:27.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T08:42:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:42:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:42:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:42:28.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:42:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:28.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:28 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T08:42:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:42:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:42:28.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:42:28.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:28.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:28.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:28.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:28.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:28 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[29355]: pgmap v814: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:42:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[29355]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T08:42:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:42:29.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:42:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[40592]: pgmap v814: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:42:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[40592]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T08:42:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:42:29.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:42:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:29 smithi172 ceph-mon[32987]: pgmap v814: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:42:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:29 smithi172 ceph-mon[32987]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T08:42:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:42:29.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:42:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:42:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:30 smithi043 ceph-mon[40592]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T08:42:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:42:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:42:30.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:42:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:30 smithi043 ceph-mon[29355]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T08:42:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:42:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:42:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:42:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:30 smithi172 ceph-mon[32987]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T08:42:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:42:30.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:42:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[29355]: pgmap v817: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:42:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[29355]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T08:42:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-102"}]: dispatch 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-102"}]: dispatch 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[40592]: pgmap v817: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[40592]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-102"}]: dispatch 2024-09-11T08:42:31.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-102"}]: dispatch 2024-09-11T08:42:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:31 smithi172 ceph-mon[32987]: pgmap v817: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:42:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:31 smithi172 ceph-mon[32987]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T08:42:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-6"}]: dispatch 2024-09-11T08:42:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2132377829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-102"}]: dispatch 2024-09-11T08:42:31.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-6", "tierpool": "test-rados-api-smithi043-79312-102"}]: dispatch 2024-09-11T08:42:32.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:32 smithi043 ceph-mon[29355]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T08:42:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:32 smithi043 ceph-mon[40592]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T08:42:32.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:32 smithi172 ceph-mon[32987]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T08:42:33.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:33] "GET /metrics HTTP/1.1" 200 228696 "" "Prometheus/2.43.0" 2024-09-11T08:42:33.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[40592]: pgmap v820: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:33.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[40592]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T08:42:33.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:33.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:33.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[29355]: pgmap v820: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[29355]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T08:42:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:33.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:33 smithi172 ceph-mon[32987]: pgmap v820: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:33 smithi172 ceph-mon[32987]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T08:42:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:33.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:42:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:34 smithi172 ceph-mon[32987]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T08:42:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:34 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-79312-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:42:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:34 smithi043 ceph-mon[40592]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T08:42:35.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:34 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-79312-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:35.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:42:35.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:34 smithi043 ceph-mon[29355]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T08:42:35.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:34 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-79312-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:42:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:42:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:36.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:42:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:42:36.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:35 smithi172 ceph-mon[32987]: pgmap v823: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:36.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:35 smithi172 ceph-mon[32987]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T08:42:36.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-79312-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:42:36.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:35 smithi172 ceph-mon[32987]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T08:42:36.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:35 smithi043 ceph-mon[40592]: pgmap v823: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:36.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:35 smithi043 ceph-mon[40592]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T08:42:36.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-79312-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:42:36.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:35 smithi043 ceph-mon[40592]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T08:42:36.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:35 smithi043 ceph-mon[29355]: pgmap v823: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:36.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:35 smithi043 ceph-mon[29355]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T08:42:36.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-79312-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:42:36.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:35 smithi043 ceph-mon[29355]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T08:42:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:37 smithi043 ceph-mon[40592]: pgmap v826: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:37 smithi043 ceph-mon[40592]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T08:42:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:37 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:37 smithi043 ceph-mon[29355]: pgmap v826: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:37 smithi043 ceph-mon[29355]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T08:42:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:37 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:37 smithi172 ceph-mon[32987]: pgmap v826: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:37 smithi172 ceph-mon[32987]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T08:42:37.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:37 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:38 smithi043 ceph-mon[40592]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T08:42:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:38.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:38 smithi043 ceph-mon[29355]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T08:42:38.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:38.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:38 smithi172 ceph-mon[32987]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T08:42:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:38.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:39 smithi043 ceph-mon[40592]: pgmap v829: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:39 smithi043 ceph-mon[40592]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T08:42:39.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:39 smithi043 ceph-mon[29355]: pgmap v829: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:39.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:39.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:39 smithi043 ceph-mon[29355]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T08:42:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:39 smithi172 ceph-mon[32987]: pgmap v829: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:39 smithi172 ceph-mon[32987]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T08:42:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[40592]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T08:42:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[40592]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T08:42:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[29355]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T08:42:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:40 smithi043 ceph-mon[29355]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T08:42:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:40 smithi172 ceph-mon[32987]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T08:42:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:40 smithi172 ceph-mon[32987]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T08:42:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[40592]: pgmap v832: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[40592]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T08:42:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[29355]: pgmap v832: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[29355]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T08:42:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:41 smithi172 ceph-mon[32987]: pgmap v832: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:41.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:41.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:41 smithi172 ceph-mon[32987]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T08:42:41.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:41.985 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[40592]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T08:42:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[29355]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T08:42:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-107", "overlaypool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:42 smithi172 ceph-mon[32987]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T08:42:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:43.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:43] "GET /metrics HTTP/1.1" 200 231113 "" "Prometheus/2.43.0" 2024-09-11T08:42:43.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[40592]: pgmap v835: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:43.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:43.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]': finished 2024-09-11T08:42:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[40592]: osdmap e535: 8 total, 8 up, 8 in 2024-09-11T08:42:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[29355]: pgmap v835: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]': finished 2024-09-11T08:42:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:43 smithi043 ceph-mon[29355]: osdmap e535: 8 total, 8 up, 8 in 2024-09-11T08:42:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:43 smithi172 ceph-mon[32987]: pgmap v835: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:42:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:43 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:43 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-107-cache", "mode": "writeback"}]': finished 2024-09-11T08:42:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:43 smithi172 ceph-mon[32987]: osdmap e535: 8 total, 8 up, 8 in 2024-09-11T08:42:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]: dispatch 2024-09-11T08:42:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]: dispatch 2024-09-11T08:42:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]: dispatch 2024-09-11T08:42:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]: dispatch 2024-09-11T08:42:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]: dispatch 2024-09-11T08:42:44.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]: dispatch 2024-09-11T08:42:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]: dispatch 2024-09-11T08:42:44.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]: dispatch 2024-09-11T08:42:44.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]: dispatch 2024-09-11T08:42:44.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]: dispatch 2024-09-11T08:42:44.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]: dispatch 2024-09-11T08:42:44.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:44 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]: dispatch 2024-09-11T08:42:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:44 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]: dispatch 2024-09-11T08:42:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]: dispatch 2024-09-11T08:42:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:44 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]: dispatch 2024-09-11T08:42:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:44 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]: dispatch 2024-09-11T08:42:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:44 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]: dispatch 2024-09-11T08:42:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:44 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]: dispatch 2024-09-11T08:42:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:42:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:45 smithi172 ceph-mon[32987]: pgmap v838: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]': finished 2024-09-11T08:42:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:45 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]': finished 2024-09-11T08:42:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:45 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]': finished 2024-09-11T08:42:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:45 smithi172 ceph-mon[32987]: osdmap e536: 8 total, 8 up, 8 in 2024-09-11T08:42:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[40592]: pgmap v838: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]': finished 2024-09-11T08:42:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]': finished 2024-09-11T08:42:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]': finished 2024-09-11T08:42:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[40592]: osdmap e536: 8 total, 8 up, 8 in 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[29355]: pgmap v838: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-107"}]': finished 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.10", "id": [7, 3]}]': finished 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.18", "id": [7, 4]}]': finished 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[29355]: osdmap e536: 8 total, 8 up, 8 in 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1338220495' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]: dispatch 2024-09-11T08:42:46.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:42:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:42:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:46 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:46 smithi043 ceph-mon[29355]: osdmap e537: 8 total, 8 up, 8 in 2024-09-11T08:42:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:46 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:46.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:46 smithi043 ceph-mon[40592]: osdmap e537: 8 total, 8 up, 8 in 2024-09-11T08:42:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:46 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-107", "tierpool": "test-rados-api-smithi043-79312-107-cache"}]': finished 2024-09-11T08:42:46.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:46 smithi172 ceph-mon[32987]: osdmap e537: 8 total, 8 up, 8 in 2024-09-11T08:42:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:47 smithi043 ceph-mon[40592]: pgmap v841: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:47 smithi043 ceph-mon[40592]: osdmap e538: 8 total, 8 up, 8 in 2024-09-11T08:42:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:47 smithi043 ceph-mon[29355]: pgmap v841: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:47 smithi043 ceph-mon[29355]: osdmap e538: 8 total, 8 up, 8 in 2024-09-11T08:42:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:47 smithi172 ceph-mon[32987]: pgmap v841: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:47 smithi172 ceph-mon[32987]: osdmap e538: 8 total, 8 up, 8 in 2024-09-11T08:42:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[40592]: osdmap e539: 8 total, 8 up, 8 in 2024-09-11T08:42:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[29355]: osdmap e539: 8 total, 8 up, 8 in 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:48 smithi172 ceph-mon[32987]: osdmap e539: 8 total, 8 up, 8 in 2024-09-11T08:42:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:42:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[40592]: pgmap v844: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:42:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[40592]: osdmap e540: 8 total, 8 up, 8 in 2024-09-11T08:42:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:50.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[29355]: pgmap v844: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:42:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[29355]: osdmap e540: 8 total, 8 up, 8 in 2024-09-11T08:42:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:50.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:50 smithi172 ceph-mon[32987]: pgmap v844: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:42:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:50 smithi172 ceph-mon[32987]: osdmap e540: 8 total, 8 up, 8 in 2024-09-11T08:42:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:50.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:42:51.384 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:51 smithi172 ceph-mon[32987]: osdmap e541: 8 total, 8 up, 8 in 2024-09-11T08:42:51.384 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:51 smithi172 ceph-mon[32987]: pgmap v847: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:51.385 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:42:51.385 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:51 smithi172 ceph-mon[32987]: osdmap e542: 8 total, 8 up, 8 in 2024-09-11T08:42:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:51 smithi043 ceph-mon[40592]: osdmap e541: 8 total, 8 up, 8 in 2024-09-11T08:42:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:51 smithi043 ceph-mon[40592]: pgmap v847: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:42:51.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:51 smithi043 ceph-mon[40592]: osdmap e542: 8 total, 8 up, 8 in 2024-09-11T08:42:51.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:51 smithi043 ceph-mon[29355]: osdmap e541: 8 total, 8 up, 8 in 2024-09-11T08:42:51.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:51 smithi043 ceph-mon[29355]: pgmap v847: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:51.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:42:51.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:51 smithi043 ceph-mon[29355]: osdmap e542: 8 total, 8 up, 8 in 2024-09-11T08:42:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:52 smithi043 ceph-mon[40592]: osdmap e543: 8 total, 8 up, 8 in 2024-09-11T08:42:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:52.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:52 smithi043 ceph-mon[29355]: osdmap e543: 8 total, 8 up, 8 in 2024-09-11T08:42:52.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:52.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:52 smithi172 ceph-mon[32987]: osdmap e543: 8 total, 8 up, 8 in 2024-09-11T08:42:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:42:53.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:53] "GET /metrics HTTP/1.1" 200 231113 "" "Prometheus/2.43.0" 2024-09-11T08:42:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[40592]: pgmap v850: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[40592]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[40592]: osdmap e544: 8 total, 8 up, 8 in 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[29355]: pgmap v850: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[29355]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[29355]: osdmap e544: 8 total, 8 up, 8 in 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:53.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:53 smithi172 ceph-mon[32987]: pgmap v850: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:53 smithi172 ceph-mon[32987]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:42:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:53 smithi172 ceph-mon[32987]: osdmap e544: 8 total, 8 up, 8 in 2024-09-11T08:42:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:42:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:54 smithi043 ceph-mon[40592]: osdmap e545: 8 total, 8 up, 8 in 2024-09-11T08:42:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:54.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:42:54.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:54 smithi043 ceph-mon[29355]: osdmap e545: 8 total, 8 up, 8 in 2024-09-11T08:42:54.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:54.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:42:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:54 smithi172 ceph-mon[32987]: osdmap e545: 8 total, 8 up, 8 in 2024-09-11T08:42:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:54.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:42:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:42:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:42:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:55 smithi172 ceph-mon[32987]: pgmap v853: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:42:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:55 smithi172 ceph-mon[32987]: osdmap e546: 8 total, 8 up, 8 in 2024-09-11T08:42:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[40592]: pgmap v853: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:42:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[40592]: osdmap e546: 8 total, 8 up, 8 in 2024-09-11T08:42:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:55.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[29355]: pgmap v853: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-109", "overlaypool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:42:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[29355]: osdmap e546: 8 total, 8 up, 8 in 2024-09-11T08:42:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:42:56.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:42:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:42:56.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:56.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]': finished 2024-09-11T08:42:56.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[40592]: osdmap e547: 8 total, 8 up, 8 in 2024-09-11T08:42:56.633 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:56.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:56.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]': finished 2024-09-11T08:42:56.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[29355]: osdmap e547: 8 total, 8 up, 8 in 2024-09-11T08:42:56.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:56.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:56 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:42:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-109-cache", "mode": "writeback"}]': finished 2024-09-11T08:42:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:56 smithi172 ceph-mon[32987]: osdmap e547: 8 total, 8 up, 8 in 2024-09-11T08:42:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:56.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[29355]: pgmap v856: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:42:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[29355]: osdmap e548: 8 total, 8 up, 8 in 2024-09-11T08:42:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:57.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:42:57.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:57.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[40592]: pgmap v856: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:57.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:42:57.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[40592]: osdmap e548: 8 total, 8 up, 8 in 2024-09-11T08:42:57.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:57.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:57.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:57.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:57.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:57 smithi172 ceph-mon[32987]: pgmap v856: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:57.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:42:57.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:57 smithi172 ceph-mon[32987]: osdmap e548: 8 total, 8 up, 8 in 2024-09-11T08:42:57.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:57.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:42:57.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:57 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:57.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: osdmap e549: 8 total, 8 up, 8 in 2024-09-11T08:42:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:58.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:58.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:42:58.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: osdmap e550: 8 total, 8 up, 8 in 2024-09-11T08:42:58.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:42:58.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:42:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: osdmap e549: 8 total, 8 up, 8 in 2024-09-11T08:42:59.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: osdmap e550: 8 total, 8 up, 8 in 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: osdmap e549: 8 total, 8 up, 8 in 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:59.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:42:59.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:42:59.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:42:59.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:42:59.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: osdmap e550: 8 total, 8 up, 8 in 2024-09-11T08:42:59.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:42:59.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:42:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:59 smithi172 ceph-mon[32987]: pgmap v859: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:42:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-11T08:42:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:42:59 smithi172 ceph-mon[32987]: osdmap e551: 8 total, 8 up, 8 in 2024-09-11T08:43:00.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:59 smithi043 ceph-mon[40592]: pgmap v859: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:00.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-11T08:43:00.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:42:59 smithi043 ceph-mon[40592]: osdmap e551: 8 total, 8 up, 8 in 2024-09-11T08:43:00.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:59 smithi043 ceph-mon[29355]: pgmap v859: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:00.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-11T08:43:00.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:42:59 smithi043 ceph-mon[29355]: osdmap e551: 8 total, 8 up, 8 in 2024-09-11T08:43:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:00 smithi172 ceph-mon[32987]: pgmap v862: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:01.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:01.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:01.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:00 smithi043 ceph-mon[40592]: pgmap v862: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:01.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:01.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:01.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:00 smithi043 ceph-mon[29355]: pgmap v862: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:01 smithi172 ceph-mon[32987]: osdmap e552: 8 total, 8 up, 8 in 2024-09-11T08:43:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:43:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:43:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:01 smithi043 ceph-mon[40592]: osdmap e552: 8 total, 8 up, 8 in 2024-09-11T08:43:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:43:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:43:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:01 smithi043 ceph-mon[29355]: osdmap e552: 8 total, 8 up, 8 in 2024-09-11T08:43:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:43:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]: dispatch 2024-09-11T08:43:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:43:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:02 smithi172 ceph-mon[32987]: osdmap e553: 8 total, 8 up, 8 in 2024-09-11T08:43:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:02 smithi172 ceph-mon[32987]: pgmap v865: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:43:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:02 smithi043 ceph-mon[29355]: osdmap e553: 8 total, 8 up, 8 in 2024-09-11T08:43:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:02 smithi043 ceph-mon[29355]: pgmap v865: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-109", "tierpool": "test-rados-api-smithi043-79312-109-cache"}]': finished 2024-09-11T08:43:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:02 smithi043 ceph-mon[40592]: osdmap e553: 8 total, 8 up, 8 in 2024-09-11T08:43:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:02 smithi043 ceph-mon[40592]: pgmap v865: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:03 smithi172 ceph-mon[32987]: osdmap e554: 8 total, 8 up, 8 in 2024-09-11T08:43:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:03 smithi172 ceph-mon[32987]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:03 smithi043 ceph-mon[40592]: osdmap e554: 8 total, 8 up, 8 in 2024-09-11T08:43:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:03 smithi043 ceph-mon[40592]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:04.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:03 smithi043 ceph-mon[29355]: osdmap e554: 8 total, 8 up, 8 in 2024-09-11T08:43:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:03 smithi043 ceph-mon[29355]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:04.145 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:43:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:03] "GET /metrics HTTP/1.1" 200 231113 "" "Prometheus/2.43.0" 2024-09-11T08:43:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:04 smithi172 ceph-mon[32987]: osdmap e555: 8 total, 8 up, 8 in 2024-09-11T08:43:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:04 smithi172 ceph-mon[32987]: pgmap v868: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:05.045 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:04 smithi043 ceph-mon[40592]: osdmap e555: 8 total, 8 up, 8 in 2024-09-11T08:43:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:05.046 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:04 smithi043 ceph-mon[40592]: pgmap v868: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:04 smithi043 ceph-mon[29355]: osdmap e555: 8 total, 8 up, 8 in 2024-09-11T08:43:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:04 smithi043 ceph-mon[29355]: pgmap v868: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:05.683 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:43:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:05 smithi172 ceph-mon[32987]: osdmap e556: 8 total, 8 up, 8 in 2024-09-11T08:43:05.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:05.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:05 smithi043 ceph-mon[40592]: osdmap e556: 8 total, 8 up, 8 in 2024-09-11T08:43:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:06.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:05 smithi043 ceph-mon[29355]: osdmap e556: 8 total, 8 up, 8 in 2024-09-11T08:43:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1410702347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]: dispatch 2024-09-11T08:43:06.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:43:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:43:06.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:06.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:06 smithi172 ceph-mon[32987]: osdmap e557: 8 total, 8 up, 8 in 2024-09-11T08:43:06.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:06 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:06.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:06 smithi172 ceph-mon[32987]: pgmap v871: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:06 smithi043 ceph-mon[40592]: osdmap e557: 8 total, 8 up, 8 in 2024-09-11T08:43:07.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:06 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:07.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:06 smithi043 ceph-mon[40592]: pgmap v871: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-109"}]': finished 2024-09-11T08:43:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:06 smithi043 ceph-mon[29355]: osdmap e557: 8 total, 8 up, 8 in 2024-09-11T08:43:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:06 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:06 smithi043 ceph-mon[29355]: pgmap v871: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:07 smithi172 ceph-mon[32987]: osdmap e558: 8 total, 8 up, 8 in 2024-09-11T08:43:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:43:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:07 smithi043 ceph-mon[40592]: osdmap e558: 8 total, 8 up, 8 in 2024-09-11T08:43:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:43:08.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:07 smithi043 ceph-mon[29355]: osdmap e558: 8 total, 8 up, 8 in 2024-09-11T08:43:08.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:43:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:43:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:08 smithi172 ceph-mon[32987]: osdmap e559: 8 total, 8 up, 8 in 2024-09-11T08:43:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:43:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:08 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:08 smithi172 ceph-mon[32987]: pgmap v874: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:43:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[40592]: osdmap e559: 8 total, 8 up, 8 in 2024-09-11T08:43:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:43:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[40592]: pgmap v874: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:43:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[29355]: osdmap e559: 8 total, 8 up, 8 in 2024-09-11T08:43:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]: dispatch 2024-09-11T08:43:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:08 smithi043 ceph-mon[29355]: pgmap v874: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:43:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:09 smithi172 ceph-mon[32987]: osdmap e560: 8 total, 8 up, 8 in 2024-09-11T08:43:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:43:09.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:43:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:43:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[40592]: osdmap e560: 8 total, 8 up, 8 in 2024-09-11T08:43:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/412978114' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-79312-104"}]': finished 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[29355]: osdmap e560: 8 total, 8 up, 8 in 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:43:10.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:43:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:43:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:10 smithi172 ceph-mon[32987]: osdmap e561: 8 total, 8 up, 8 in 2024-09-11T08:43:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:10.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:10 smithi172 ceph-mon[32987]: pgmap v877: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:43:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:43:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[40592]: osdmap e561: 8 total, 8 up, 8 in 2024-09-11T08:43:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[40592]: pgmap v877: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:43:11.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-79312-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:43:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[29355]: osdmap e561: 8 total, 8 up, 8 in 2024-09-11T08:43:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:10 smithi043 ceph-mon[29355]: pgmap v877: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:43:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:11 smithi172 ceph-mon[32987]: osdmap e562: 8 total, 8 up, 8 in 2024-09-11T08:43:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:11 smithi043 ceph-mon[40592]: osdmap e562: 8 total, 8 up, 8 in 2024-09-11T08:43:12.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:11 smithi043 ceph-mon[29355]: osdmap e562: 8 total, 8 up, 8 in 2024-09-11T08:43:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:12 smithi172 ceph-mon[32987]: osdmap e563: 8 total, 8 up, 8 in 2024-09-11T08:43:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:12 smithi172 ceph-mon[32987]: pgmap v880: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:12 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[40592]: osdmap e563: 8 total, 8 up, 8 in 2024-09-11T08:43:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[40592]: pgmap v880: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:13.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-79312-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[29355]: osdmap e563: 8 total, 8 up, 8 in 2024-09-11T08:43:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[29355]: pgmap v880: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:43:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:13] "GET /metrics HTTP/1.1" 200 226255 "" "Prometheus/2.43.0" 2024-09-11T08:43:14.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:13 smithi043 ceph-mon[40592]: osdmap e564: 8 total, 8 up, 8 in 2024-09-11T08:43:14.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:14.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:13 smithi043 ceph-mon[29355]: osdmap e564: 8 total, 8 up, 8 in 2024-09-11T08:43:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:13 smithi172 ceph-mon[32987]: osdmap e564: 8 total, 8 up, 8 in 2024-09-11T08:43:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[40592]: osdmap e565: 8 total, 8 up, 8 in 2024-09-11T08:43:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[40592]: pgmap v883: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[29355]: osdmap e565: 8 total, 8 up, 8 in 2024-09-11T08:43:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:14 smithi043 ceph-mon[29355]: pgmap v883: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:14 smithi172 ceph-mon[32987]: osdmap e565: 8 total, 8 up, 8 in 2024-09-11T08:43:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:14 smithi172 ceph-mon[32987]: pgmap v883: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:43:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:16.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:16.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:15 smithi172 ceph-mon[32987]: osdmap e566: 8 total, 8 up, 8 in 2024-09-11T08:43:16.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:16.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:15 smithi043 ceph-mon[40592]: osdmap e566: 8 total, 8 up, 8 in 2024-09-11T08:43:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:15 smithi043 ceph-mon[29355]: osdmap e566: 8 total, 8 up, 8 in 2024-09-11T08:43:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:16.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:43:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:43:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]': finished 2024-09-11T08:43:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[40592]: osdmap e567: 8 total, 8 up, 8 in 2024-09-11T08:43:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[40592]: pgmap v886: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]': finished 2024-09-11T08:43:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[29355]: osdmap e567: 8 total, 8 up, 8 in 2024-09-11T08:43:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:16 smithi043 ceph-mon[29355]: pgmap v886: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-112"}]': finished 2024-09-11T08:43:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:16 smithi172 ceph-mon[32987]: osdmap e567: 8 total, 8 up, 8 in 2024-09-11T08:43:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:16 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:16 smithi172 ceph-mon[32987]: pgmap v886: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:18.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:18.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:17 smithi043 ceph-mon[40592]: osdmap e568: 8 total, 8 up, 8 in 2024-09-11T08:43:18.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:18.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:17 smithi043 ceph-mon[29355]: osdmap e568: 8 total, 8 up, 8 in 2024-09-11T08:43:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:18.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:17 smithi172 ceph-mon[32987]: osdmap e568: 8 total, 8 up, 8 in 2024-09-11T08:43:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:18.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]: dispatch 2024-09-11T08:43:19.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]': finished 2024-09-11T08:43:19.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:18 smithi043 ceph-mon[40592]: osdmap e569: 8 total, 8 up, 8 in 2024-09-11T08:43:19.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:18 smithi043 ceph-mon[40592]: pgmap v889: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]': finished 2024-09-11T08:43:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:18 smithi043 ceph-mon[29355]: osdmap e569: 8 total, 8 up, 8 in 2024-09-11T08:43:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:18 smithi043 ceph-mon[29355]: pgmap v889: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-112"}]': finished 2024-09-11T08:43:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:18 smithi172 ceph-mon[32987]: osdmap e569: 8 total, 8 up, 8 in 2024-09-11T08:43:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:18 smithi172 ceph-mon[32987]: pgmap v889: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:19 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:19 smithi043 ceph-mon[40592]: osdmap e570: 8 total, 8 up, 8 in 2024-09-11T08:43:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:19 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:19 smithi043 ceph-mon[29355]: osdmap e570: 8 total, 8 up, 8 in 2024-09-11T08:43:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:19 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:19 smithi172 ceph-mon[32987]: osdmap e570: 8 total, 8 up, 8 in 2024-09-11T08:43:21.132 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:20 smithi172 ceph-mon[32987]: osdmap e571: 8 total, 8 up, 8 in 2024-09-11T08:43:21.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:21.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:21.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:20 smithi172 ceph-mon[32987]: pgmap v892: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:20 smithi043 ceph-mon[40592]: osdmap e571: 8 total, 8 up, 8 in 2024-09-11T08:43:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:20 smithi043 ceph-mon[40592]: pgmap v892: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:20 smithi043 ceph-mon[29355]: osdmap e571: 8 total, 8 up, 8 in 2024-09-11T08:43:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:20 smithi043 ceph-mon[29355]: pgmap v892: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:21 smithi043 ceph-mon[40592]: osdmap e572: 8 total, 8 up, 8 in 2024-09-11T08:43:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:21 smithi043 ceph-mon[29355]: osdmap e572: 8 total, 8 up, 8 in 2024-09-11T08:43:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:21 smithi172 ceph-mon[32987]: osdmap e572: 8 total, 8 up, 8 in 2024-09-11T08:43:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[40592]: osdmap e573: 8 total, 8 up, 8 in 2024-09-11T08:43:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[40592]: pgmap v895: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[29355]: osdmap e573: 8 total, 8 up, 8 in 2024-09-11T08:43:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:22 smithi043 ceph-mon[29355]: pgmap v895: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:22 smithi172 ceph-mon[32987]: osdmap e573: 8 total, 8 up, 8 in 2024-09-11T08:43:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:22 smithi172 ceph-mon[32987]: pgmap v895: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:23] "GET /metrics HTTP/1.1" 200 226255 "" "Prometheus/2.43.0" 2024-09-11T08:43:24.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]': finished 2024-09-11T08:43:24.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-mon[40592]: osdmap e574: 8 total, 8 up, 8 in 2024-09-11T08:43:24.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]: dispatch 2024-09-11T08:43:24.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]: dispatch 2024-09-11T08:43:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]': finished 2024-09-11T08:43:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-mon[29355]: osdmap e574: 8 total, 8 up, 8 in 2024-09-11T08:43:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]: dispatch 2024-09-11T08:43:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]: dispatch 2024-09-11T08:43:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-114"}]': finished 2024-09-11T08:43:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:23 smithi172 ceph-mon[32987]: osdmap e574: 8 total, 8 up, 8 in 2024-09-11T08:43:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]: dispatch 2024-09-11T08:43:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]: dispatch 2024-09-11T08:43:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]': finished 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[40592]: osdmap e575: 8 total, 8 up, 8 in 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[40592]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]': finished 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[29355]: osdmap e575: 8 total, 8 up, 8 in 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:25.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:25.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:24 smithi043 ceph-mon[29355]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:24 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-114", "mode": "writeback"}]': finished 2024-09-11T08:43:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:24 smithi172 ceph-mon[32987]: osdmap e575: 8 total, 8 up, 8 in 2024-09-11T08:43:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:24 smithi172 ceph-mon[32987]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:43:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:26.144 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:26.144 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:25 smithi172 ceph-mon[32987]: osdmap e576: 8 total, 8 up, 8 in 2024-09-11T08:43:26.144 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:26.144 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:26.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:26.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:25 smithi043 ceph-mon[40592]: osdmap e576: 8 total, 8 up, 8 in 2024-09-11T08:43:26.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:26.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:26.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:26.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:25 smithi043 ceph-mon[29355]: osdmap e576: 8 total, 8 up, 8 in 2024-09-11T08:43:26.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:26.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]: dispatch 2024-09-11T08:43:26.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:43:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:43:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:43:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]': finished 2024-09-11T08:43:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[40592]: osdmap e577: 8 total, 8 up, 8 in 2024-09-11T08:43:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[40592]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:27.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:43:27.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]': finished 2024-09-11T08:43:27.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[29355]: osdmap e577: 8 total, 8 up, 8 in 2024-09-11T08:43:27.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:27.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:26 smithi043 ceph-mon[29355]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:26 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:43:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-114"}]': finished 2024-09-11T08:43:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:26 smithi172 ceph-mon[32987]: osdmap e577: 8 total, 8 up, 8 in 2024-09-11T08:43:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:26 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:26 smithi172 ceph-mon[32987]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:43:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:27 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:27 smithi043 ceph-mon[40592]: osdmap e578: 8 total, 8 up, 8 in 2024-09-11T08:43:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:28.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:27 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:28.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:27 smithi043 ceph-mon[29355]: osdmap e578: 8 total, 8 up, 8 in 2024-09-11T08:43:28.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:27 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:27 smithi172 ceph-mon[32987]: osdmap e578: 8 total, 8 up, 8 in 2024-09-11T08:43:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[40592]: osdmap e579: 8 total, 8 up, 8 in 2024-09-11T08:43:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[40592]: pgmap v904: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[29355]: osdmap e579: 8 total, 8 up, 8 in 2024-09-11T08:43:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[29355]: pgmap v904: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:29.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:28 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:28 smithi172 ceph-mon[32987]: osdmap e579: 8 total, 8 up, 8 in 2024-09-11T08:43:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:28 smithi172 ceph-mon[32987]: pgmap v904: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:28 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[40592]: osdmap e580: 8 total, 8 up, 8 in 2024-09-11T08:43:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:30.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[40592]: osdmap e581: 8 total, 8 up, 8 in 2024-09-11T08:43:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[29355]: osdmap e580: 8 total, 8 up, 8 in 2024-09-11T08:43:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:29 smithi043 ceph-mon[29355]: osdmap e581: 8 total, 8 up, 8 in 2024-09-11T08:43:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:29 smithi172 ceph-mon[32987]: osdmap e580: 8 total, 8 up, 8 in 2024-09-11T08:43:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:29 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:29 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:30.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:29 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:30.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:29 smithi172 ceph-mon[32987]: osdmap e581: 8 total, 8 up, 8 in 2024-09-11T08:43:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:31.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:30 smithi043 ceph-mon[40592]: pgmap v907: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:31.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:31.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:31.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:30 smithi043 ceph-mon[29355]: pgmap v907: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:31.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:30 smithi172 ceph-mon[32987]: pgmap v907: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:32.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:32.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:31 smithi043 ceph-mon[40592]: osdmap e582: 8 total, 8 up, 8 in 2024-09-11T08:43:32.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:32.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:32.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:32.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:31 smithi043 ceph-mon[29355]: osdmap e582: 8 total, 8 up, 8 in 2024-09-11T08:43:32.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:32.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:32.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:32.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:31 smithi172 ceph-mon[32987]: osdmap e582: 8 total, 8 up, 8 in 2024-09-11T08:43:32.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:32.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]': finished 2024-09-11T08:43:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[40592]: osdmap e583: 8 total, 8 up, 8 in 2024-09-11T08:43:33.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]: dispatch 2024-09-11T08:43:33.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]: dispatch 2024-09-11T08:43:33.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[40592]: pgmap v910: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]': finished 2024-09-11T08:43:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[29355]: osdmap e583: 8 total, 8 up, 8 in 2024-09-11T08:43:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]: dispatch 2024-09-11T08:43:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]: dispatch 2024-09-11T08:43:33.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:32 smithi043 ceph-mon[29355]: pgmap v910: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-116"}]': finished 2024-09-11T08:43:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:32 smithi172 ceph-mon[32987]: osdmap e583: 8 total, 8 up, 8 in 2024-09-11T08:43:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]: dispatch 2024-09-11T08:43:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]: dispatch 2024-09-11T08:43:33.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:32 smithi172 ceph-mon[32987]: pgmap v910: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:33] "GET /metrics HTTP/1.1" 200 226307 "" "Prometheus/2.43.0" 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]': finished 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[40592]: osdmap e584: 8 total, 8 up, 8 in 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.1"}]: dispatch 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[40592]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.1"}]: dispatch 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]': finished 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[29355]: osdmap e584: 8 total, 8 up, 8 in 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.1"}]: dispatch 2024-09-11T08:43:34.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[29355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.1"}]: dispatch 2024-09-11T08:43:34.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:33 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:33 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-116", "mode": "writeback"}]': finished 2024-09-11T08:43:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:33 smithi172 ceph-mon[32987]: osdmap e584: 8 total, 8 up, 8 in 2024-09-11T08:43:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.1"}]: dispatch 2024-09-11T08:43:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:33 smithi172 ceph-mon[32987]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.1"}]: dispatch 2024-09-11T08:43:34.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:33 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:43:35.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:34 smithi043 ceph-mon[40592]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-11T08:43:35.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:34 smithi043 ceph-mon[29355]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-11T08:43:35.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:34 smithi172 ceph-mon[32987]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-11T08:43:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:43:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:36.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:35 smithi172 ceph-mon[32987]: 244.1 scrub starts 2024-09-11T08:43:36.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:35 smithi172 ceph-mon[32987]: 244.1 scrub ok 2024-09-11T08:43:36.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:43:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:43:36.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:35 smithi043 ceph-mon[40592]: 244.1 scrub starts 2024-09-11T08:43:36.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:35 smithi043 ceph-mon[40592]: 244.1 scrub ok 2024-09-11T08:43:36.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:35 smithi043 ceph-mon[29355]: 244.1 scrub starts 2024-09-11T08:43:36.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:35 smithi043 ceph-mon[29355]: 244.1 scrub ok 2024-09-11T08:43:37.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:36 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:37.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:36 smithi172 ceph-mon[32987]: pgmap v913: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:43:37.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:36 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:37.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:36 smithi043 ceph-mon[40592]: pgmap v913: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:43:37.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:36 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:37.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:36 smithi043 ceph-mon[29355]: pgmap v913: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:43:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:39 smithi043 ceph-mon[40592]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 970 B/s wr, 2 op/s 2024-09-11T08:43:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:39 smithi043 ceph-mon[29355]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 970 B/s wr, 2 op/s 2024-09-11T08:43:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:39 smithi172 ceph-mon[32987]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 970 B/s wr, 2 op/s 2024-09-11T08:43:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:41 smithi043 ceph-mon[40592]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 854 B/s wr, 2 op/s 2024-09-11T08:43:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:41 smithi043 ceph-mon[29355]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 854 B/s wr, 2 op/s 2024-09-11T08:43:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:41 smithi172 ceph-mon[32987]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 854 B/s wr, 2 op/s 2024-09-11T08:43:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:43 smithi043 ceph-mon[40592]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-11T08:43:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:43 smithi043 ceph-mon[29355]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-11T08:43:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:43 smithi172 ceph-mon[32987]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-11T08:43:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:43:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:43] "GET /metrics HTTP/1.1" 200 228725 "" "Prometheus/2.43.0" 2024-09-11T08:43:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:45 smithi043 ceph-mon[40592]: pgmap v917: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 630 B/s wr, 2 op/s 2024-09-11T08:43:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:45 smithi043 ceph-mon[29355]: pgmap v917: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 630 B/s wr, 2 op/s 2024-09-11T08:43:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:45 smithi172 ceph-mon[32987]: pgmap v917: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 630 B/s wr, 2 op/s 2024-09-11T08:43:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:43:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:46.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:43:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:43:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:47 smithi043 ceph-mon[40592]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:43:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:47 smithi043 ceph-mon[29355]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:43:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:47 smithi172 ceph-mon[32987]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:43:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:49 smithi043 ceph-mon[29355]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:49 smithi043 ceph-mon[29355]: osdmap e585: 8 total, 8 up, 8 in 2024-09-11T08:43:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:49 smithi043 ceph-mon[40592]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:49 smithi043 ceph-mon[40592]: osdmap e585: 8 total, 8 up, 8 in 2024-09-11T08:43:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:49.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:49 smithi172 ceph-mon[32987]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:49 smithi172 ceph-mon[32987]: osdmap e585: 8 total, 8 up, 8 in 2024-09-11T08:43:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:50 smithi043 ceph-mon[40592]: osdmap e586: 8 total, 8 up, 8 in 2024-09-11T08:43:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:50 smithi043 ceph-mon[29355]: osdmap e586: 8 total, 8 up, 8 in 2024-09-11T08:43:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:43:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:50 smithi172 ceph-mon[32987]: osdmap e586: 8 total, 8 up, 8 in 2024-09-11T08:43:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]: dispatch 2024-09-11T08:43:51.632 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:51 smithi172 ceph-mon[32987]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-11T08:43:51.632 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:51 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:43:51.632 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]': finished 2024-09-11T08:43:51.632 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:51 smithi172 ceph-mon[32987]: osdmap e587: 8 total, 8 up, 8 in 2024-09-11T08:43:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:51 smithi043 ceph-mon[40592]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-11T08:43:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:51 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:43:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]': finished 2024-09-11T08:43:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:51 smithi043 ceph-mon[40592]: osdmap e587: 8 total, 8 up, 8 in 2024-09-11T08:43:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:51 smithi043 ceph-mon[29355]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-11T08:43:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:51 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:43:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-116"}]': finished 2024-09-11T08:43:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:51 smithi043 ceph-mon[29355]: osdmap e587: 8 total, 8 up, 8 in 2024-09-11T08:43:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:52 smithi043 ceph-mon[40592]: osdmap e588: 8 total, 8 up, 8 in 2024-09-11T08:43:52.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:52 smithi043 ceph-mon[29355]: osdmap e588: 8 total, 8 up, 8 in 2024-09-11T08:43:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:52 smithi172 ceph-mon[32987]: osdmap e588: 8 total, 8 up, 8 in 2024-09-11T08:43:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-mon[40592]: pgmap v925: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-11T08:43:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-mon[40592]: osdmap e589: 8 total, 8 up, 8 in 2024-09-11T08:43:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:53.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:53.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-mon[29355]: pgmap v925: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-11T08:43:53.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-mon[29355]: osdmap e589: 8 total, 8 up, 8 in 2024-09-11T08:43:53.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:53.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:53 smithi172 ceph-mon[32987]: pgmap v925: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-11T08:43:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:53 smithi172 ceph-mon[32987]: osdmap e589: 8 total, 8 up, 8 in 2024-09-11T08:43:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:43:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:43:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:53] "GET /metrics HTTP/1.1" 200 228725 "" "Prometheus/2.43.0" 2024-09-11T08:43:54.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:54.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:54 smithi043 ceph-mon[29355]: osdmap e590: 8 total, 8 up, 8 in 2024-09-11T08:43:54.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:54.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:54 smithi043 ceph-mon[40592]: osdmap e590: 8 total, 8 up, 8 in 2024-09-11T08:43:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:43:54.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:54 smithi172 ceph-mon[32987]: osdmap e590: 8 total, 8 up, 8 in 2024-09-11T08:43:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:55 smithi043 ceph-mon[40592]: pgmap v928: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:55 smithi043 ceph-mon[40592]: osdmap e591: 8 total, 8 up, 8 in 2024-09-11T08:43:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:55 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:55 smithi043 ceph-mon[29355]: pgmap v928: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:55 smithi043 ceph-mon[29355]: osdmap e591: 8 total, 8 up, 8 in 2024-09-11T08:43:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:55 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:43:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:43:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:55 smithi172 ceph-mon[32987]: pgmap v928: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:55 smithi172 ceph-mon[32987]: osdmap e591: 8 total, 8 up, 8 in 2024-09-11T08:43:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:55 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:43:56.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:43:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:43:56.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:56 smithi172 ceph-mon[32987]: osdmap e592: 8 total, 8 up, 8 in 2024-09-11T08:43:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:43:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:43:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:56 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]': finished 2024-09-11T08:43:56.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:56 smithi172 ceph-mon[32987]: osdmap e593: 8 total, 8 up, 8 in 2024-09-11T08:43:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[40592]: osdmap e592: 8 total, 8 up, 8 in 2024-09-11T08:43:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:43:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:43:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]': finished 2024-09-11T08:43:56.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[40592]: osdmap e593: 8 total, 8 up, 8 in 2024-09-11T08:43:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:43:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[29355]: osdmap e592: 8 total, 8 up, 8 in 2024-09-11T08:43:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:43:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:43:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-118"}]': finished 2024-09-11T08:43:56.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:56 smithi043 ceph-mon[29355]: osdmap e593: 8 total, 8 up, 8 in 2024-09-11T08:43:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: pgmap v931: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]: dispatch 2024-09-11T08:43:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]: dispatch 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]': finished 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[40592]: osdmap e594: 8 total, 8 up, 8 in 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: pgmap v931: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]: dispatch 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]: dispatch 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]': finished 2024-09-11T08:43:57.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:57 smithi043 ceph-mon[29355]: osdmap e594: 8 total, 8 up, 8 in 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: pgmap v931: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]: dispatch 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]: dispatch 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-118", "mode": "writeback"}]': finished 2024-09-11T08:43:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:57 smithi172 ceph-mon[32987]: osdmap e594: 8 total, 8 up, 8 in 2024-09-11T08:43:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:59 smithi172 ceph-mon[32987]: osdmap e595: 8 total, 8 up, 8 in 2024-09-11T08:43:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:59.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:43:59 smithi172 ceph-mon[32987]: pgmap v935: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:59 smithi043 ceph-mon[40592]: osdmap e595: 8 total, 8 up, 8 in 2024-09-11T08:43:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:43:59 smithi043 ceph-mon[40592]: pgmap v935: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:43:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:59 smithi043 ceph-mon[29355]: osdmap e595: 8 total, 8 up, 8 in 2024-09-11T08:43:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:43:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:43:59 smithi043 ceph-mon[29355]: pgmap v935: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:00 smithi172 ceph-mon[32987]: osdmap e596: 8 total, 8 up, 8 in 2024-09-11T08:44:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:44:00.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:44:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:00 smithi043 ceph-mon[40592]: osdmap e596: 8 total, 8 up, 8 in 2024-09-11T08:44:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:44:00.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:44:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:00.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:00 smithi043 ceph-mon[29355]: osdmap e596: 8 total, 8 up, 8 in 2024-09-11T08:44:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:44:00.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]: dispatch 2024-09-11T08:44:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:01 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]': finished 2024-09-11T08:44:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:01 smithi172 ceph-mon[32987]: osdmap e597: 8 total, 8 up, 8 in 2024-09-11T08:44:01.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:01 smithi172 ceph-mon[32987]: pgmap v938: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 525 B/s wr, 2 op/s 2024-09-11T08:44:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:01 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]': finished 2024-09-11T08:44:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:01 smithi043 ceph-mon[40592]: osdmap e597: 8 total, 8 up, 8 in 2024-09-11T08:44:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:01 smithi043 ceph-mon[40592]: pgmap v938: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 525 B/s wr, 2 op/s 2024-09-11T08:44:01.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:01 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:01.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-118"}]': finished 2024-09-11T08:44:01.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:01 smithi043 ceph-mon[29355]: osdmap e597: 8 total, 8 up, 8 in 2024-09-11T08:44:01.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:01 smithi043 ceph-mon[29355]: pgmap v938: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 525 B/s wr, 2 op/s 2024-09-11T08:44:02.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:02 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:02.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:02 smithi172 ceph-mon[32987]: osdmap e598: 8 total, 8 up, 8 in 2024-09-11T08:44:02.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:02 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:02.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:02 smithi043 ceph-mon[40592]: osdmap e598: 8 total, 8 up, 8 in 2024-09-11T08:44:02.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:02 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:02.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:02 smithi043 ceph-mon[29355]: osdmap e598: 8 total, 8 up, 8 in 2024-09-11T08:44:03.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:03 smithi172 ceph-mon[32987]: osdmap e599: 8 total, 8 up, 8 in 2024-09-11T08:44:03.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:03.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:03.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:03 smithi172 ceph-mon[32987]: pgmap v941: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:44:03.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:03.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:03 smithi172 ceph-mon[32987]: osdmap e600: 8 total, 8 up, 8 in 2024-09-11T08:44:03.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:03.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[40592]: osdmap e599: 8 total, 8 up, 8 in 2024-09-11T08:44:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[40592]: pgmap v941: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:44:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[40592]: osdmap e600: 8 total, 8 up, 8 in 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[29355]: osdmap e599: 8 total, 8 up, 8 in 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[29355]: pgmap v941: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[29355]: osdmap e600: 8 total, 8 up, 8 in 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:03.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:44:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:03] "GET /metrics HTTP/1.1" 200 228710 "" "Prometheus/2.43.0" 2024-09-11T08:44:05.411 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:05 smithi172 ceph-mon[32987]: osdmap e601: 8 total, 8 up, 8 in 2024-09-11T08:44:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:05.412 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:05 smithi172 ceph-mon[32987]: pgmap v944: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:05.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:05.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[40592]: osdmap e601: 8 total, 8 up, 8 in 2024-09-11T08:44:05.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:05.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:05.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[40592]: pgmap v944: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:05.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:05.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[29355]: osdmap e601: 8 total, 8 up, 8 in 2024-09-11T08:44:05.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:05.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:05.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:05 smithi043 ceph-mon[29355]: pgmap v944: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:44:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:06.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:44:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:44:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]': finished 2024-09-11T08:44:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:06 smithi172 ceph-mon[32987]: osdmap e602: 8 total, 8 up, 8 in 2024-09-11T08:44:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]: dispatch 2024-09-11T08:44:06.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]: dispatch 2024-09-11T08:44:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]': finished 2024-09-11T08:44:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:06 smithi043 ceph-mon[40592]: osdmap e602: 8 total, 8 up, 8 in 2024-09-11T08:44:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]: dispatch 2024-09-11T08:44:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]: dispatch 2024-09-11T08:44:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-120"}]': finished 2024-09-11T08:44:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:06 smithi043 ceph-mon[29355]: osdmap e602: 8 total, 8 up, 8 in 2024-09-11T08:44:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]: dispatch 2024-09-11T08:44:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]: dispatch 2024-09-11T08:44:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:07 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]': finished 2024-09-11T08:44:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:07 smithi172 ceph-mon[32987]: osdmap e603: 8 total, 8 up, 8 in 2024-09-11T08:44:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:07.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:07.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:07.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:07 smithi172 ceph-mon[32987]: pgmap v947: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]': finished 2024-09-11T08:44:07.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[40592]: osdmap e603: 8 total, 8 up, 8 in 2024-09-11T08:44:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:07.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[40592]: pgmap v947: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-120", "mode": "writeback"}]': finished 2024-09-11T08:44:07.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[29355]: osdmap e603: 8 total, 8 up, 8 in 2024-09-11T08:44:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:07.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:07 smithi043 ceph-mon[29355]: pgmap v947: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:08.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:08.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:08 smithi172 ceph-mon[32987]: osdmap e604: 8 total, 8 up, 8 in 2024-09-11T08:44:08.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:08.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:08.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:08 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[40592]: osdmap e604: 8 total, 8 up, 8 in 2024-09-11T08:44:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:08.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[29355]: osdmap e604: 8 total, 8 up, 8 in 2024-09-11T08:44:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]: dispatch 2024-09-11T08:44:08.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:08 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:09.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:09 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:09.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]': finished 2024-09-11T08:44:09.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:09 smithi172 ceph-mon[32987]: osdmap e605: 8 total, 8 up, 8 in 2024-09-11T08:44:09.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:09 smithi172 ceph-mon[32987]: pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:09 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]': finished 2024-09-11T08:44:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:09 smithi043 ceph-mon[40592]: osdmap e605: 8 total, 8 up, 8 in 2024-09-11T08:44:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:09 smithi043 ceph-mon[40592]: pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:09 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-120"}]': finished 2024-09-11T08:44:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:09 smithi043 ceph-mon[29355]: osdmap e605: 8 total, 8 up, 8 in 2024-09-11T08:44:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:09 smithi043 ceph-mon[29355]: pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:10.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:10 smithi172 ceph-mon[32987]: osdmap e606: 8 total, 8 up, 8 in 2024-09-11T08:44:10.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:10 smithi043 ceph-mon[40592]: osdmap e606: 8 total, 8 up, 8 in 2024-09-11T08:44:10.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:10 smithi043 ceph-mon[29355]: osdmap e606: 8 total, 8 up, 8 in 2024-09-11T08:44:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:11 smithi172 ceph-mon[32987]: osdmap e607: 8 total, 8 up, 8 in 2024-09-11T08:44:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:11 smithi172 ceph-mon[32987]: pgmap v953: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:11.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:11 smithi172 ceph-mon[32987]: osdmap e608: 8 total, 8 up, 8 in 2024-09-11T08:44:11.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[29355]: osdmap e607: 8 total, 8 up, 8 in 2024-09-11T08:44:11.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:11.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:11.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[29355]: pgmap v953: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:11.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:11.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[29355]: osdmap e608: 8 total, 8 up, 8 in 2024-09-11T08:44:11.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[40592]: osdmap e607: 8 total, 8 up, 8 in 2024-09-11T08:44:11.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:11.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:11.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[40592]: pgmap v953: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:11.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:11.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:11 smithi043 ceph-mon[40592]: osdmap e608: 8 total, 8 up, 8 in 2024-09-11T08:44:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:12 smithi172 ceph-mon[32987]: osdmap e609: 8 total, 8 up, 8 in 2024-09-11T08:44:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:12.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[40592]: osdmap e609: 8 total, 8 up, 8 in 2024-09-11T08:44:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[29355]: osdmap e609: 8 total, 8 up, 8 in 2024-09-11T08:44:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:13 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:13 smithi172 ceph-mon[32987]: pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:13 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]': finished 2024-09-11T08:44:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:13 smithi172 ceph-mon[32987]: osdmap e610: 8 total, 8 up, 8 in 2024-09-11T08:44:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]: dispatch 2024-09-11T08:44:13.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]: dispatch 2024-09-11T08:44:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[40592]: pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]': finished 2024-09-11T08:44:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[40592]: osdmap e610: 8 total, 8 up, 8 in 2024-09-11T08:44:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]: dispatch 2024-09-11T08:44:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]: dispatch 2024-09-11T08:44:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[29355]: pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-122"}]': finished 2024-09-11T08:44:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[29355]: osdmap e610: 8 total, 8 up, 8 in 2024-09-11T08:44:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]: dispatch 2024-09-11T08:44:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]: dispatch 2024-09-11T08:44:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:44:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:13] "GET /metrics HTTP/1.1" 200 228741 "" "Prometheus/2.43.0" 2024-09-11T08:44:14.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:14 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:14.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]': finished 2024-09-11T08:44:14.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:14 smithi172 ceph-mon[32987]: osdmap e611: 8 total, 8 up, 8 in 2024-09-11T08:44:14.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:14 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:14.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]': finished 2024-09-11T08:44:14.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:14 smithi043 ceph-mon[29355]: osdmap e611: 8 total, 8 up, 8 in 2024-09-11T08:44:14.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:14 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:14.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-122", "mode": "writeback"}]': finished 2024-09-11T08:44:14.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:14 smithi043 ceph-mon[40592]: osdmap e611: 8 total, 8 up, 8 in 2024-09-11T08:44:15.484 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:44:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:15.484 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:15.484 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:15.485 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:15 smithi172 ceph-mon[32987]: pgmap v959: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:15 smithi043 ceph-mon[40592]: pgmap v959: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:15.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:15.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:15.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:15 smithi043 ceph-mon[29355]: pgmap v959: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:16.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:44:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:44:16.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:16.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:16 smithi172 ceph-mon[32987]: osdmap e612: 8 total, 8 up, 8 in 2024-09-11T08:44:16.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:16.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:16 smithi043 ceph-mon[40592]: osdmap e612: 8 total, 8 up, 8 in 2024-09-11T08:44:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:16 smithi043 ceph-mon[29355]: osdmap e612: 8 total, 8 up, 8 in 2024-09-11T08:44:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]: dispatch 2024-09-11T08:44:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:17 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]': finished 2024-09-11T08:44:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:17 smithi172 ceph-mon[32987]: osdmap e613: 8 total, 8 up, 8 in 2024-09-11T08:44:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:17 smithi172 ceph-mon[32987]: pgmap v962: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]': finished 2024-09-11T08:44:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[40592]: osdmap e613: 8 total, 8 up, 8 in 2024-09-11T08:44:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[40592]: pgmap v962: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:17.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-122"}]': finished 2024-09-11T08:44:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[29355]: osdmap e613: 8 total, 8 up, 8 in 2024-09-11T08:44:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:17 smithi043 ceph-mon[29355]: pgmap v962: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:18.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:18 smithi172 ceph-mon[32987]: osdmap e614: 8 total, 8 up, 8 in 2024-09-11T08:44:18.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:18 smithi043 ceph-mon[40592]: osdmap e614: 8 total, 8 up, 8 in 2024-09-11T08:44:18.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:18 smithi043 ceph-mon[29355]: osdmap e614: 8 total, 8 up, 8 in 2024-09-11T08:44:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:19 smithi043 ceph-mon[40592]: osdmap e615: 8 total, 8 up, 8 in 2024-09-11T08:44:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:19 smithi043 ceph-mon[40592]: pgmap v965: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:19.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:19 smithi043 ceph-mon[29355]: osdmap e615: 8 total, 8 up, 8 in 2024-09-11T08:44:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:19 smithi043 ceph-mon[29355]: pgmap v965: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:19 smithi172 ceph-mon[32987]: osdmap e615: 8 total, 8 up, 8 in 2024-09-11T08:44:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:19 smithi172 ceph-mon[32987]: pgmap v965: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:20.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:20.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:20 smithi043 ceph-mon[40592]: osdmap e616: 8 total, 8 up, 8 in 2024-09-11T08:44:20.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:20.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:20 smithi043 ceph-mon[29355]: osdmap e616: 8 total, 8 up, 8 in 2024-09-11T08:44:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:20 smithi172 ceph-mon[32987]: osdmap e616: 8 total, 8 up, 8 in 2024-09-11T08:44:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:21 smithi172 ceph-mon[32987]: pgmap v967: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-11T08:44:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:21 smithi172 ceph-mon[32987]: osdmap e617: 8 total, 8 up, 8 in 2024-09-11T08:44:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:21 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:21.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[40592]: pgmap v967: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[40592]: osdmap e617: 8 total, 8 up, 8 in 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[29355]: pgmap v967: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-11T08:44:21.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[29355]: osdmap e617: 8 total, 8 up, 8 in 2024-09-11T08:44:21.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:21.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:21.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:22 smithi043 ceph-mon[29355]: osdmap e618: 8 total, 8 up, 8 in 2024-09-11T08:44:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:22 smithi043 ceph-mon[40592]: osdmap e618: 8 total, 8 up, 8 in 2024-09-11T08:44:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:22 smithi172 ceph-mon[32987]: osdmap e618: 8 total, 8 up, 8 in 2024-09-11T08:44:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[29355]: pgmap v970: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]': finished 2024-09-11T08:44:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[29355]: osdmap e619: 8 total, 8 up, 8 in 2024-09-11T08:44:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]: dispatch 2024-09-11T08:44:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]: dispatch 2024-09-11T08:44:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[40592]: pgmap v970: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]': finished 2024-09-11T08:44:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[40592]: osdmap e619: 8 total, 8 up, 8 in 2024-09-11T08:44:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]: dispatch 2024-09-11T08:44:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]: dispatch 2024-09-11T08:44:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:23 smithi172 ceph-mon[32987]: pgmap v970: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-124"}]': finished 2024-09-11T08:44:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:23 smithi172 ceph-mon[32987]: osdmap e619: 8 total, 8 up, 8 in 2024-09-11T08:44:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]: dispatch 2024-09-11T08:44:23.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]: dispatch 2024-09-11T08:44:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:44:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:23] "GET /metrics HTTP/1.1" 200 228741 "" "Prometheus/2.43.0" 2024-09-11T08:44:24.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:24 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:24.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]': finished 2024-09-11T08:44:24.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:24 smithi043 ceph-mon[29355]: osdmap e620: 8 total, 8 up, 8 in 2024-09-11T08:44:24.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:24 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:24.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]': finished 2024-09-11T08:44:24.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:24 smithi043 ceph-mon[40592]: osdmap e620: 8 total, 8 up, 8 in 2024-09-11T08:44:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:24 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-124", "mode": "writeback"}]': finished 2024-09-11T08:44:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:24 smithi172 ceph-mon[32987]: osdmap e620: 8 total, 8 up, 8 in 2024-09-11T08:44:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:25 smithi043 ceph-mon[40592]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-11T08:44:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:25 smithi043 ceph-mon[40592]: osdmap e621: 8 total, 8 up, 8 in 2024-09-11T08:44:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:25 smithi043 ceph-mon[29355]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-11T08:44:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:25 smithi043 ceph-mon[29355]: osdmap e621: 8 total, 8 up, 8 in 2024-09-11T08:44:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:25.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:44:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:25 smithi172 ceph-mon[32987]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-11T08:44:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:25 smithi172 ceph-mon[32987]: osdmap e621: 8 total, 8 up, 8 in 2024-09-11T08:44:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:25.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:26 smithi172 ceph-mon[32987]: osdmap e622: 8 total, 8 up, 8 in 2024-09-11T08:44:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:26.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:26.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:26 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:26.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]': finished 2024-09-11T08:44:26.482 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:26 smithi172 ceph-mon[32987]: osdmap e623: 8 total, 8 up, 8 in 2024-09-11T08:44:26.482 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:44:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:44:26.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:26.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[40592]: osdmap e622: 8 total, 8 up, 8 in 2024-09-11T08:44:26.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]': finished 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[40592]: osdmap e623: 8 total, 8 up, 8 in 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[29355]: osdmap e622: 8 total, 8 up, 8 in 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]: dispatch 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-124"}]': finished 2024-09-11T08:44:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:26 smithi043 ceph-mon[29355]: osdmap e623: 8 total, 8 up, 8 in 2024-09-11T08:44:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:27 smithi043 ceph-mon[40592]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-11T08:44:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:27.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:27.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:27 smithi043 ceph-mon[29355]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-11T08:44:27.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:27 smithi172 ceph-mon[32987]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-11T08:44:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:28.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:28 smithi043 ceph-mon[40592]: osdmap e624: 8 total, 8 up, 8 in 2024-09-11T08:44:28.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:28 smithi043 ceph-mon[29355]: osdmap e624: 8 total, 8 up, 8 in 2024-09-11T08:44:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:28 smithi172 ceph-mon[32987]: osdmap e624: 8 total, 8 up, 8 in 2024-09-11T08:44:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[40592]: pgmap v979: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[40592]: osdmap e625: 8 total, 8 up, 8 in 2024-09-11T08:44:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:29.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[29355]: pgmap v979: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:29.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[29355]: osdmap e625: 8 total, 8 up, 8 in 2024-09-11T08:44:29.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:29.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:29.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:29 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:29 smithi172 ceph-mon[32987]: pgmap v979: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:29 smithi172 ceph-mon[32987]: osdmap e625: 8 total, 8 up, 8 in 2024-09-11T08:44:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:29 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[40592]: osdmap e626: 8 total, 8 up, 8 in 2024-09-11T08:44:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[29355]: osdmap e626: 8 total, 8 up, 8 in 2024-09-11T08:44:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:30 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:30 smithi172 ceph-mon[32987]: osdmap e626: 8 total, 8 up, 8 in 2024-09-11T08:44:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:30 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:30 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:30 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[40592]: pgmap v982: 204 pgs: 11 creating+peering, 21 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:44:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[40592]: osdmap e627: 8 total, 8 up, 8 in 2024-09-11T08:44:31.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:31.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[29355]: pgmap v982: 204 pgs: 11 creating+peering, 21 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:44:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[29355]: osdmap e627: 8 total, 8 up, 8 in 2024-09-11T08:44:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:31 smithi172 ceph-mon[32987]: pgmap v982: 204 pgs: 11 creating+peering, 21 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:44:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:31 smithi172 ceph-mon[32987]: osdmap e627: 8 total, 8 up, 8 in 2024-09-11T08:44:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:32.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]': finished 2024-09-11T08:44:32.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:32 smithi043 ceph-mon[40592]: osdmap e628: 8 total, 8 up, 8 in 2024-09-11T08:44:32.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]: dispatch 2024-09-11T08:44:32.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]: dispatch 2024-09-11T08:44:32.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]': finished 2024-09-11T08:44:32.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:32 smithi043 ceph-mon[29355]: osdmap e628: 8 total, 8 up, 8 in 2024-09-11T08:44:32.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]: dispatch 2024-09-11T08:44:32.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]: dispatch 2024-09-11T08:44:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-126"}]': finished 2024-09-11T08:44:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:32 smithi172 ceph-mon[32987]: osdmap e628: 8 total, 8 up, 8 in 2024-09-11T08:44:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]: dispatch 2024-09-11T08:44:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]: dispatch 2024-09-11T08:44:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: pgmap v985: 204 pgs: 11 creating+peering, 21 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]': finished 2024-09-11T08:44:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: osdmap e629: 8 total, 8 up, 8 in 2024-09-11T08:44:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: osdmap e630: 8 total, 8 up, 8 in 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: pgmap v985: 204 pgs: 11 creating+peering, 21 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]': finished 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: osdmap e629: 8 total, 8 up, 8 in 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:33.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: osdmap e630: 8 total, 8 up, 8 in 2024-09-11T08:44:33.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:33.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: pgmap v985: 204 pgs: 11 creating+peering, 21 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-126", "mode": "writeback"}]': finished 2024-09-11T08:44:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: osdmap e629: 8 total, 8 up, 8 in 2024-09-11T08:44:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: osdmap e630: 8 total, 8 up, 8 in 2024-09-11T08:44:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]: dispatch 2024-09-11T08:44:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:44:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:33] "GET /metrics HTTP/1.1" 200 228741 "" "Prometheus/2.43.0" 2024-09-11T08:44:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:34 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:34 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]': finished 2024-09-11T08:44:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:34 smithi043 ceph-mon[29355]: osdmap e631: 8 total, 8 up, 8 in 2024-09-11T08:44:34.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:34 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:34.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:34 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]': finished 2024-09-11T08:44:34.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:34 smithi043 ceph-mon[40592]: osdmap e631: 8 total, 8 up, 8 in 2024-09-11T08:44:34.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:34 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:34.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:34 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-126"}]': finished 2024-09-11T08:44:34.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:34 smithi172 ceph-mon[32987]: osdmap e631: 8 total, 8 up, 8 in 2024-09-11T08:44:35.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-11T08:44:35.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:44:35.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9067 ms) 2024-09-11T08:44:35.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-11T08:44:35.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:44:35.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6032 ms) 2024-09-11T08:44:35.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5026 ms) 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25458 ms) 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14049 ms) 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12073 ms) 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-11T08:44:35.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000012 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12069 ms) 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6038 ms) 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (561796 ms total) 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1014 ms) 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11059 ms) 2024-09-11T08:44:35.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18095 ms) 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30170 ms total) 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7048 ms) 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-11T08:44:35.162 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8057 ms) 2024-09-11T08:44:35.163 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrub... 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24489 ms) 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9852 ms) 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8014 ms) 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8049 ms) 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10067 ms) 2024-09-11T08:44:35.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-11T08:44:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:35 smithi172 ceph-mon[32987]: pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 3 op/s 2024-09-11T08:44:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:35 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:35 smithi172 ceph-mon[32987]: osdmap e632: 8 total, 8 up, 8 in 2024-09-11T08:44:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:44:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:35 smithi043 ceph-mon[29355]: pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 3 op/s 2024-09-11T08:44:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:35 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:35 smithi043 ceph-mon[29355]: osdmap e632: 8 total, 8 up, 8 in 2024-09-11T08:44:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:35 smithi043 ceph-mon[40592]: pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 3 op/s 2024-09-11T08:44:35.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:35 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:35.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:35 smithi043 ceph-mon[40592]: osdmap e632: 8 total, 8 up, 8 in 2024-09-11T08:44:36.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:44:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:44:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:37 smithi172 ceph-mon[32987]: osdmap e633: 8 total, 8 up, 8 in 2024-09-11T08:44:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:37 smithi172 ceph-mon[32987]: pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 261 B/s wr, 1 op/s 2024-09-11T08:44:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[29355]: osdmap e633: 8 total, 8 up, 8 in 2024-09-11T08:44:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:37.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[29355]: pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 261 B/s wr, 1 op/s 2024-09-11T08:44:37.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[40592]: osdmap e633: 8 total, 8 up, 8 in 2024-09-11T08:44:37.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:37.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:37.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:37.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:37 smithi043 ceph-mon[40592]: pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 261 B/s wr, 1 op/s 2024-09-11T08:44:38.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:38.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:38 smithi172 ceph-mon[32987]: osdmap e634: 8 total, 8 up, 8 in 2024-09-11T08:44:38.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:38.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:38 smithi043 ceph-mon[40592]: osdmap e634: 8 total, 8 up, 8 in 2024-09-11T08:44:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:38.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:38 smithi043 ceph-mon[29355]: osdmap e634: 8 total, 8 up, 8 in 2024-09-11T08:44:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:39 smithi172 ceph-mon[32987]: osdmap e635: 8 total, 8 up, 8 in 2024-09-11T08:44:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:39.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:39 smithi172 ceph-mon[32987]: pgmap v995: 204 pgs: 27 unknown, 177 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[40592]: osdmap e635: 8 total, 8 up, 8 in 2024-09-11T08:44:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[40592]: pgmap v995: 204 pgs: 27 unknown, 177 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[29355]: osdmap e635: 8 total, 8 up, 8 in 2024-09-11T08:44:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:39 smithi043 ceph-mon[29355]: pgmap v995: 204 pgs: 27 unknown, 177 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]': finished 2024-09-11T08:44:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:40 smithi172 ceph-mon[32987]: osdmap e636: 8 total, 8 up, 8 in 2024-09-11T08:44:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]: dispatch 2024-09-11T08:44:40.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]: dispatch 2024-09-11T08:44:40.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]': finished 2024-09-11T08:44:40.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:40 smithi043 ceph-mon[29355]: osdmap e636: 8 total, 8 up, 8 in 2024-09-11T08:44:40.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]: dispatch 2024-09-11T08:44:40.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]: dispatch 2024-09-11T08:44:40.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-128"}]': finished 2024-09-11T08:44:40.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:40 smithi043 ceph-mon[40592]: osdmap e636: 8 total, 8 up, 8 in 2024-09-11T08:44:40.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]: dispatch 2024-09-11T08:44:40.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]: dispatch 2024-09-11T08:44:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:41 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]': finished 2024-09-11T08:44:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:41 smithi172 ceph-mon[32987]: osdmap e637: 8 total, 8 up, 8 in 2024-09-11T08:44:41.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:41 smithi172 ceph-mon[32987]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:41 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]': finished 2024-09-11T08:44:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:41 smithi043 ceph-mon[40592]: osdmap e637: 8 total, 8 up, 8 in 2024-09-11T08:44:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:41 smithi043 ceph-mon[40592]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:41 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:41.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-128", "mode": "writeback"}]': finished 2024-09-11T08:44:41.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:41 smithi043 ceph-mon[29355]: osdmap e637: 8 total, 8 up, 8 in 2024-09-11T08:44:41.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:41 smithi043 ceph-mon[29355]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-mon[40592]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1006 B/s rd, 0 op/s 2024-09-11T08:44:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-mon[29355]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1006 B/s rd, 0 op/s 2024-09-11T08:44:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:43 smithi172 ceph-mon[32987]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1006 B/s rd, 0 op/s 2024-09-11T08:44:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:43 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:44:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:44:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:43] "GET /metrics HTTP/1.1" 200 228747 "" "Prometheus/2.43.0" 2024-09-11T08:44:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:45 smithi043 ceph-mon[29355]: pgmap v1000: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 336 B/s wr, 2 op/s 2024-09-11T08:44:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:45 smithi043 ceph-mon[40592]: pgmap v1000: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 336 B/s wr, 2 op/s 2024-09-11T08:44:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:45.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:45 smithi172 ceph-mon[32987]: pgmap v1000: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 336 B/s wr, 2 op/s 2024-09-11T08:44:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:44:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:46 smithi043 ceph-mon[40592]: osdmap e638: 8 total, 8 up, 8 in 2024-09-11T08:44:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:46 smithi043 ceph-mon[29355]: osdmap e638: 8 total, 8 up, 8 in 2024-09-11T08:44:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:46.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:46 smithi172 ceph-mon[32987]: osdmap e638: 8 total, 8 up, 8 in 2024-09-11T08:44:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]: dispatch 2024-09-11T08:44:46.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:44:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:44:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[29355]: pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 289 B/s wr, 1 op/s 2024-09-11T08:44:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]': finished 2024-09-11T08:44:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[29355]: osdmap e639: 8 total, 8 up, 8 in 2024-09-11T08:44:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[40592]: pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 289 B/s wr, 1 op/s 2024-09-11T08:44:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]': finished 2024-09-11T08:44:47.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:47 smithi043 ceph-mon[40592]: osdmap e639: 8 total, 8 up, 8 in 2024-09-11T08:44:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:47 smithi172 ceph-mon[32987]: pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 289 B/s wr, 1 op/s 2024-09-11T08:44:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:47 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-128"}]': finished 2024-09-11T08:44:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:47 smithi172 ceph-mon[32987]: osdmap e639: 8 total, 8 up, 8 in 2024-09-11T08:44:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:48 smithi043 ceph-mon[40592]: osdmap e640: 8 total, 8 up, 8 in 2024-09-11T08:44:48.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:48 smithi043 ceph-mon[29355]: osdmap e640: 8 total, 8 up, 8 in 2024-09-11T08:44:48.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:48 smithi172 ceph-mon[32987]: osdmap e640: 8 total, 8 up, 8 in 2024-09-11T08:44:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:49 smithi043 ceph-mon[40592]: pgmap v1005: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:44:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:49 smithi043 ceph-mon[40592]: osdmap e641: 8 total, 8 up, 8 in 2024-09-11T08:44:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:49 smithi043 ceph-mon[29355]: pgmap v1005: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:44:49.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:49 smithi043 ceph-mon[29355]: osdmap e641: 8 total, 8 up, 8 in 2024-09-11T08:44:49.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:49.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:49 smithi172 ceph-mon[32987]: pgmap v1005: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:44:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:49 smithi172 ceph-mon[32987]: osdmap e641: 8 total, 8 up, 8 in 2024-09-11T08:44:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:50 smithi043 ceph-mon[40592]: osdmap e642: 8 total, 8 up, 8 in 2024-09-11T08:44:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:50.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:50 smithi043 ceph-mon[29355]: osdmap e642: 8 total, 8 up, 8 in 2024-09-11T08:44:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:50.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:50 smithi172 ceph-mon[32987]: osdmap e642: 8 total, 8 up, 8 in 2024-09-11T08:44:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:50.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:51.632 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:51 smithi172 ceph-mon[32987]: pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:44:51.632 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:51.632 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:51 smithi172 ceph-mon[32987]: osdmap e643: 8 total, 8 up, 8 in 2024-09-11T08:44:51.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:51 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:51.633 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[40592]: pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:44:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[40592]: osdmap e643: 8 total, 8 up, 8 in 2024-09-11T08:44:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[29355]: pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:44:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[29355]: osdmap e643: 8 total, 8 up, 8 in 2024-09-11T08:44:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:51.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]': finished 2024-09-11T08:44:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[40592]: osdmap e644: 8 total, 8 up, 8 in 2024-09-11T08:44:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]: dispatch 2024-09-11T08:44:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]: dispatch 2024-09-11T08:44:52.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]': finished 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[40592]: osdmap e645: 8 total, 8 up, 8 in 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]': finished 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[29355]: osdmap e644: 8 total, 8 up, 8 in 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]: dispatch 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]: dispatch 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]': finished 2024-09-11T08:44:52.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:52 smithi043 ceph-mon[29355]: osdmap e645: 8 total, 8 up, 8 in 2024-09-11T08:44:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-130"}]': finished 2024-09-11T08:44:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:52 smithi172 ceph-mon[32987]: osdmap e644: 8 total, 8 up, 8 in 2024-09-11T08:44:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:52 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]: dispatch 2024-09-11T08:44:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]: dispatch 2024-09-11T08:44:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:52 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:44:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:52 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-130", "mode": "writeback"}]': finished 2024-09-11T08:44:52.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:52 smithi172 ceph-mon[32987]: osdmap e645: 8 total, 8 up, 8 in 2024-09-11T08:44:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:53 smithi043 ceph-mon[40592]: pgmap v1011: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:53 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:53 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:53.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:53 smithi043 ceph-mon[29355]: pgmap v1011: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:53.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:53 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:53.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:53 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:53 smithi172 ceph-mon[32987]: pgmap v1011: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:44:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:53 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:53 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:44:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:44:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:53] "GET /metrics HTTP/1.1" 200 228747 "" "Prometheus/2.43.0" 2024-09-11T08:44:54.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:54.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:54 smithi043 ceph-mon[29355]: osdmap e646: 8 total, 8 up, 8 in 2024-09-11T08:44:54.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:54 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:54.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:54 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:54.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:54.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:54 smithi043 ceph-mon[40592]: osdmap e646: 8 total, 8 up, 8 in 2024-09-11T08:44:54.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:54 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:54.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:54 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:44:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:54 smithi172 ceph-mon[32987]: osdmap e646: 8 total, 8 up, 8 in 2024-09-11T08:44:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:54 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:54.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:54 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]: dispatch 2024-09-11T08:44:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:55 smithi043 ceph-mon[29355]: pgmap v1014: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-11T08:44:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:55 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:55 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]': finished 2024-09-11T08:44:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:55 smithi043 ceph-mon[29355]: osdmap e647: 8 total, 8 up, 8 in 2024-09-11T08:44:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:55 smithi043 ceph-mon[40592]: pgmap v1014: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-11T08:44:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:55 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:55 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]': finished 2024-09-11T08:44:55.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:55 smithi043 ceph-mon[40592]: osdmap e647: 8 total, 8 up, 8 in 2024-09-11T08:44:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:44:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:44:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:55 smithi172 ceph-mon[32987]: pgmap v1014: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-11T08:44:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:55 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:44:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:55 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-130"}]': finished 2024-09-11T08:44:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:55 smithi172 ceph-mon[32987]: osdmap e647: 8 total, 8 up, 8 in 2024-09-11T08:44:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:56 smithi043 ceph-mon[40592]: osdmap e648: 8 total, 8 up, 8 in 2024-09-11T08:44:56.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:56 smithi043 ceph-mon[29355]: osdmap e648: 8 total, 8 up, 8 in 2024-09-11T08:44:56.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:56 smithi172 ceph-mon[32987]: osdmap e648: 8 total, 8 up, 8 in 2024-09-11T08:44:56.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:44:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:44:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:57 smithi172 ceph-mon[32987]: pgmap v1017: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:57 smithi172 ceph-mon[32987]: osdmap e649: 8 total, 8 up, 8 in 2024-09-11T08:44:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:57 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:57 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:57.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[40592]: pgmap v1017: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[40592]: osdmap e649: 8 total, 8 up, 8 in 2024-09-11T08:44:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[29355]: pgmap v1017: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:44:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[29355]: osdmap e649: 8 total, 8 up, 8 in 2024-09-11T08:44:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:44:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:58 smithi172 ceph-mon[32987]: osdmap e650: 8 total, 8 up, 8 in 2024-09-11T08:44:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:58 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:58.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:58.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:58 smithi043 ceph-mon[40592]: osdmap e650: 8 total, 8 up, 8 in 2024-09-11T08:44:58.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:58.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:58 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:58.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:44:58.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:58 smithi043 ceph-mon[29355]: osdmap e650: 8 total, 8 up, 8 in 2024-09-11T08:44:58.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:58.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:58 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:44:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:59 smithi172 ceph-mon[32987]: pgmap v1020: 204 pgs: 18 unknown, 186 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:44:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:59 smithi172 ceph-mon[32987]: osdmap e651: 8 total, 8 up, 8 in 2024-09-11T08:44:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:44:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:44:59 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:44:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[40592]: pgmap v1020: 204 pgs: 18 unknown, 186 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:44:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[40592]: osdmap e651: 8 total, 8 up, 8 in 2024-09-11T08:44:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:44:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:44:59.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[29355]: pgmap v1020: 204 pgs: 18 unknown, 186 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:44:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:44:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[29355]: osdmap e651: 8 total, 8 up, 8 in 2024-09-11T08:44:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:44:59.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:44:59 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:00 smithi172 ceph-mon[32987]: osdmap e652: 8 total, 8 up, 8 in 2024-09-11T08:45:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]: dispatch 2024-09-11T08:45:00.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]: dispatch 2024-09-11T08:45:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:00 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]': finished 2024-09-11T08:45:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[40592]: osdmap e652: 8 total, 8 up, 8 in 2024-09-11T08:45:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]: dispatch 2024-09-11T08:45:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]: dispatch 2024-09-11T08:45:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]': finished 2024-09-11T08:45:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[29355]: osdmap e652: 8 total, 8 up, 8 in 2024-09-11T08:45:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]: dispatch 2024-09-11T08:45:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]: dispatch 2024-09-11T08:45:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-132", "mode": "writeback"}]': finished 2024-09-11T08:45:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:01 smithi172 ceph-mon[32987]: pgmap v1023: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:01 smithi172 ceph-mon[32987]: osdmap e653: 8 total, 8 up, 8 in 2024-09-11T08:45:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:01 smithi043 ceph-mon[40592]: pgmap v1023: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:01 smithi043 ceph-mon[40592]: osdmap e653: 8 total, 8 up, 8 in 2024-09-11T08:45:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:01 smithi043 ceph-mon[29355]: pgmap v1023: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:01 smithi043 ceph-mon[29355]: osdmap e653: 8 total, 8 up, 8 in 2024-09-11T08:45:02.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:02 smithi172 ceph-mon[32987]: osdmap e654: 8 total, 8 up, 8 in 2024-09-11T08:45:02.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:02 smithi043 ceph-mon[40592]: osdmap e654: 8 total, 8 up, 8 in 2024-09-11T08:45:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:02 smithi043 ceph-mon[29355]: osdmap e654: 8 total, 8 up, 8 in 2024-09-11T08:45:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:03 smithi172 ceph-mon[32987]: pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:03 smithi172 ceph-mon[32987]: osdmap e655: 8 total, 8 up, 8 in 2024-09-11T08:45:03.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:03 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:03.733 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:03 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-mon[29355]: pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-mon[29355]: osdmap e655: 8 total, 8 up, 8 in 2024-09-11T08:45:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-mon[40592]: pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-mon[40592]: osdmap e655: 8 total, 8 up, 8 in 2024-09-11T08:45:03.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:03.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:45:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:03] "GET /metrics HTTP/1.1" 200 228735 "" "Prometheus/2.43.0" 2024-09-11T08:45:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:04 smithi172 ceph-mon[32987]: osdmap e656: 8 total, 8 up, 8 in 2024-09-11T08:45:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:04.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:04 smithi043 ceph-mon[29355]: osdmap e656: 8 total, 8 up, 8 in 2024-09-11T08:45:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:04.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:04.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:04 smithi043 ceph-mon[40592]: osdmap e656: 8 total, 8 up, 8 in 2024-09-11T08:45:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:04.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:45:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:05 smithi172 ceph-mon[32987]: pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:45:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:05 smithi172 ceph-mon[32987]: osdmap e657: 8 total, 8 up, 8 in 2024-09-11T08:45:05.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:05 smithi043 ceph-mon[29355]: pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:45:05.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:05.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:05 smithi043 ceph-mon[29355]: osdmap e657: 8 total, 8 up, 8 in 2024-09-11T08:45:05.796 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:05 smithi043 ceph-mon[40592]: pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:45:05.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:05.797 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:05 smithi043 ceph-mon[40592]: osdmap e657: 8 total, 8 up, 8 in 2024-09-11T08:45:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:06 smithi172 ceph-mon[32987]: osdmap e658: 8 total, 8 up, 8 in 2024-09-11T08:45:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:06.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:45:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:45:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:06 smithi043 ceph-mon[40592]: osdmap e658: 8 total, 8 up, 8 in 2024-09-11T08:45:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:06.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:06.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:06 smithi043 ceph-mon[29355]: osdmap e658: 8 total, 8 up, 8 in 2024-09-11T08:45:06.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:06.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:07 smithi172 ceph-mon[32987]: pgmap v1032: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:45:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:07 smithi172 ceph-mon[32987]: osdmap e659: 8 total, 8 up, 8 in 2024-09-11T08:45:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[40592]: pgmap v1032: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:45:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[40592]: osdmap e659: 8 total, 8 up, 8 in 2024-09-11T08:45:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[29355]: pgmap v1032: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:45:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[29355]: osdmap e659: 8 total, 8 up, 8 in 2024-09-11T08:45:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:07.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]: dispatch 2024-09-11T08:45:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:08 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:08.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:08 smithi172 ceph-mon[32987]: osdmap e660: 8 total, 8 up, 8 in 2024-09-11T08:45:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:08 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:08.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:08 smithi043 ceph-mon[40592]: osdmap e660: 8 total, 8 up, 8 in 2024-09-11T08:45:08.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:08 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:08.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-132"}]': finished 2024-09-11T08:45:08.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:08 smithi043 ceph-mon[29355]: osdmap e660: 8 total, 8 up, 8 in 2024-09-11T08:45:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:09 smithi172 ceph-mon[32987]: pgmap v1035: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:09 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:09 smithi172 ceph-mon[32987]: osdmap e661: 8 total, 8 up, 8 in 2024-09-11T08:45:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:09 smithi043 ceph-mon[40592]: pgmap v1035: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:09 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:09 smithi043 ceph-mon[40592]: osdmap e661: 8 total, 8 up, 8 in 2024-09-11T08:45:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:09 smithi043 ceph-mon[29355]: pgmap v1035: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:09 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:09.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:09 smithi043 ceph-mon[29355]: osdmap e661: 8 total, 8 up, 8 in 2024-09-11T08:45:10.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[29355]: osdmap e662: 8 total, 8 up, 8 in 2024-09-11T08:45:10.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:10.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:10.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:10.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[29355]: osdmap e663: 8 total, 8 up, 8 in 2024-09-11T08:45:10.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[40592]: osdmap e662: 8 total, 8 up, 8 in 2024-09-11T08:45:10.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:10.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:10.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:10.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:10 smithi043 ceph-mon[40592]: osdmap e663: 8 total, 8 up, 8 in 2024-09-11T08:45:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:10 smithi172 ceph-mon[32987]: osdmap e662: 8 total, 8 up, 8 in 2024-09-11T08:45:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:10.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:10.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:10 smithi172 ceph-mon[32987]: osdmap e663: 8 total, 8 up, 8 in 2024-09-11T08:45:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[29355]: pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[29355]: osdmap e664: 8 total, 8 up, 8 in 2024-09-11T08:45:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:11.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[40592]: pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[40592]: osdmap e664: 8 total, 8 up, 8 in 2024-09-11T08:45:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:11.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:11 smithi172 ceph-mon[32987]: pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:11 smithi172 ceph-mon[32987]: osdmap e664: 8 total, 8 up, 8 in 2024-09-11T08:45:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:11.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]': finished 2024-09-11T08:45:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[29355]: osdmap e665: 8 total, 8 up, 8 in 2024-09-11T08:45:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]: dispatch 2024-09-11T08:45:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]: dispatch 2024-09-11T08:45:12.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:12.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]': finished 2024-09-11T08:45:12.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[40592]: osdmap e665: 8 total, 8 up, 8 in 2024-09-11T08:45:12.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]: dispatch 2024-09-11T08:45:12.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]: dispatch 2024-09-11T08:45:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-134"}]': finished 2024-09-11T08:45:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:12 smithi172 ceph-mon[32987]: osdmap e665: 8 total, 8 up, 8 in 2024-09-11T08:45:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]: dispatch 2024-09-11T08:45:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]: dispatch 2024-09-11T08:45:13.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:13] "GET /metrics HTTP/1.1" 200 228751 "" "Prometheus/2.43.0" 2024-09-11T08:45:13.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[40592]: pgmap v1041: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:13.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]': finished 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[40592]: osdmap e666: 8 total, 8 up, 8 in 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[29355]: pgmap v1041: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]': finished 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[29355]: osdmap e666: 8 total, 8 up, 8 in 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:13.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:13.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:13 smithi172 ceph-mon[32987]: pgmap v1041: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:13.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:13 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:13.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-134", "mode": "writeback"}]': finished 2024-09-11T08:45:13.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:13 smithi172 ceph-mon[32987]: osdmap e666: 8 total, 8 up, 8 in 2024-09-11T08:45:13.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:13.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:15.481 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:45:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:15.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:15.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:15 smithi172 ceph-mon[32987]: osdmap e667: 8 total, 8 up, 8 in 2024-09-11T08:45:15.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:15.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:15.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:15 smithi172 ceph-mon[32987]: pgmap v1045: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-11T08:45:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[40592]: osdmap e667: 8 total, 8 up, 8 in 2024-09-11T08:45:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:15.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[40592]: pgmap v1045: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-11T08:45:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[29355]: osdmap e667: 8 total, 8 up, 8 in 2024-09-11T08:45:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]: dispatch 2024-09-11T08:45:15.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:15 smithi043 ceph-mon[29355]: pgmap v1045: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-11T08:45:16.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:16 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:16.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]': finished 2024-09-11T08:45:16.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:16 smithi172 ceph-mon[32987]: osdmap e668: 8 total, 8 up, 8 in 2024-09-11T08:45:16.481 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:45:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:45:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:16 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]': finished 2024-09-11T08:45:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:16 smithi043 ceph-mon[40592]: osdmap e668: 8 total, 8 up, 8 in 2024-09-11T08:45:16.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:16 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-134"}]': finished 2024-09-11T08:45:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:16 smithi043 ceph-mon[29355]: osdmap e668: 8 total, 8 up, 8 in 2024-09-11T08:45:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:17 smithi172 ceph-mon[32987]: osdmap e669: 8 total, 8 up, 8 in 2024-09-11T08:45:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:17 smithi172 ceph-mon[32987]: pgmap v1048: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-11T08:45:17.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:17 smithi043 ceph-mon[40592]: osdmap e669: 8 total, 8 up, 8 in 2024-09-11T08:45:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:17 smithi043 ceph-mon[40592]: pgmap v1048: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-11T08:45:17.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:17 smithi043 ceph-mon[29355]: osdmap e669: 8 total, 8 up, 8 in 2024-09-11T08:45:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:17 smithi043 ceph-mon[29355]: pgmap v1048: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-11T08:45:17.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:18.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:18 smithi172 ceph-mon[32987]: osdmap e670: 8 total, 8 up, 8 in 2024-09-11T08:45:18.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:18.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:18.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:18 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:18.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:18 smithi043 ceph-mon[29355]: osdmap e670: 8 total, 8 up, 8 in 2024-09-11T08:45:18.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:18.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:18.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:18 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:18 smithi043 ceph-mon[40592]: osdmap e670: 8 total, 8 up, 8 in 2024-09-11T08:45:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:18.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:18 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:19 smithi172 ceph-mon[32987]: osdmap e671: 8 total, 8 up, 8 in 2024-09-11T08:45:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:19 smithi172 ceph-mon[32987]: pgmap v1051: 204 pgs: 15 unknown, 189 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[40592]: osdmap e671: 8 total, 8 up, 8 in 2024-09-11T08:45:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:19.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[40592]: pgmap v1051: 204 pgs: 15 unknown, 189 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[29355]: osdmap e671: 8 total, 8 up, 8 in 2024-09-11T08:45:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:19.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:19 smithi043 ceph-mon[29355]: pgmap v1051: 204 pgs: 15 unknown, 189 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:20.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:20.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:20 smithi172 ceph-mon[32987]: osdmap e672: 8 total, 8 up, 8 in 2024-09-11T08:45:20.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:20.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:20.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:20.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:20 smithi043 ceph-mon[40592]: osdmap e672: 8 total, 8 up, 8 in 2024-09-11T08:45:20.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:20.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:20 smithi043 ceph-mon[29355]: osdmap e672: 8 total, 8 up, 8 in 2024-09-11T08:45:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:20.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]': finished 2024-09-11T08:45:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:21 smithi172 ceph-mon[32987]: osdmap e673: 8 total, 8 up, 8 in 2024-09-11T08:45:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]: dispatch 2024-09-11T08:45:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]: dispatch 2024-09-11T08:45:21.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:21 smithi172 ceph-mon[32987]: pgmap v1054: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:21.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]': finished 2024-09-11T08:45:21.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[29355]: osdmap e673: 8 total, 8 up, 8 in 2024-09-11T08:45:21.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]: dispatch 2024-09-11T08:45:21.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]: dispatch 2024-09-11T08:45:21.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[29355]: pgmap v1054: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:21.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-136"}]': finished 2024-09-11T08:45:21.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[40592]: osdmap e673: 8 total, 8 up, 8 in 2024-09-11T08:45:21.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]: dispatch 2024-09-11T08:45:21.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]: dispatch 2024-09-11T08:45:21.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:21 smithi043 ceph-mon[40592]: pgmap v1054: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:22 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]': finished 2024-09-11T08:45:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:22 smithi172 ceph-mon[32987]: osdmap e674: 8 total, 8 up, 8 in 2024-09-11T08:45:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:22.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:22.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]': finished 2024-09-11T08:45:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[29355]: osdmap e674: 8 total, 8 up, 8 in 2024-09-11T08:45:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:22.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:45:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-136", "mode": "writeback"}]': finished 2024-09-11T08:45:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[40592]: osdmap e674: 8 total, 8 up, 8 in 2024-09-11T08:45:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:22.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:23.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:23.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:23 smithi172 ceph-mon[32987]: osdmap e675: 8 total, 8 up, 8 in 2024-09-11T08:45:23.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:23.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:23.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:23 smithi172 ceph-mon[32987]: pgmap v1057: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[40592]: osdmap e675: 8 total, 8 up, 8 in 2024-09-11T08:45:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:23.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[40592]: pgmap v1057: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:45:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[29355]: osdmap e675: 8 total, 8 up, 8 in 2024-09-11T08:45:23.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:23.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]: dispatch 2024-09-11T08:45:23.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-mon[29355]: pgmap v1057: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:45:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:45:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:23] "GET /metrics HTTP/1.1" 200 228751 "" "Prometheus/2.43.0" 2024-09-11T08:45:24.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:24 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:24.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]': finished 2024-09-11T08:45:24.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:24 smithi172 ceph-mon[32987]: osdmap e676: 8 total, 8 up, 8 in 2024-09-11T08:45:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:24 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]': finished 2024-09-11T08:45:24.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:24 smithi043 ceph-mon[40592]: osdmap e676: 8 total, 8 up, 8 in 2024-09-11T08:45:24.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:24 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:45:24.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-136"}]': finished 2024-09-11T08:45:24.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:24 smithi043 ceph-mon[29355]: osdmap e676: 8 total, 8 up, 8 in 2024-09-11T08:45:25.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:25 smithi172 ceph-mon[32987]: osdmap e677: 8 total, 8 up, 8 in 2024-09-11T08:45:25.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:25 smithi172 ceph-mon[32987]: pgmap v1060: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:45:25.481 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:45:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:25 smithi043 ceph-mon[40592]: osdmap e677: 8 total, 8 up, 8 in 2024-09-11T08:45:25.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:25 smithi043 ceph-mon[40592]: pgmap v1060: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:45:25.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:25 smithi043 ceph-mon[29355]: osdmap e677: 8 total, 8 up, 8 in 2024-09-11T08:45:25.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:25 smithi043 ceph-mon[29355]: pgmap v1060: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:45:26.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:26 smithi043 ceph-mon[40592]: osdmap e678: 8 total, 8 up, 8 in 2024-09-11T08:45:26.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:26.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:26.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:26 smithi043 ceph-mon[29355]: osdmap e678: 8 total, 8 up, 8 in 2024-09-11T08:45:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:26.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:26.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:45:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:45:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:26 smithi172 ceph-mon[32987]: osdmap e678: 8 total, 8 up, 8 in 2024-09-11T08:45:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[40592]: osdmap e679: 8 total, 8 up, 8 in 2024-09-11T08:45:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[40592]: pgmap v1063: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:45:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:27.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[29355]: osdmap e679: 8 total, 8 up, 8 in 2024-09-11T08:45:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[29355]: pgmap v1063: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:45:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:27.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:27 smithi172 ceph-mon[32987]: osdmap e679: 8 total, 8 up, 8 in 2024-09-11T08:45:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:27 smithi172 ceph-mon[32987]: pgmap v1063: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:45:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:28.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:28.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:28 smithi043 ceph-mon[40592]: osdmap e680: 8 total, 8 up, 8 in 2024-09-11T08:45:28.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:28 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:45:28.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:28 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:45:28.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:28.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:28 smithi043 ceph-mon[29355]: osdmap e680: 8 total, 8 up, 8 in 2024-09-11T08:45:28.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:28 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:45:28.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:28 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:45:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:28 smithi172 ceph-mon[32987]: osdmap e680: 8 total, 8 up, 8 in 2024-09-11T08:45:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:28 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:45:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:28 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:45:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:45:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[40592]: osdmap e681: 8 total, 8 up, 8 in 2024-09-11T08:45:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:45:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:45:29.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[40592]: pgmap v1066: 204 pgs: 12 unknown, 192 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:29.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:45:29.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[29355]: osdmap e681: 8 total, 8 up, 8 in 2024-09-11T08:45:29.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:45:29.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:45:29.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:29 smithi043 ceph-mon[29355]: pgmap v1066: 204 pgs: 12 unknown, 192 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:45:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:29 smithi172 ceph-mon[32987]: osdmap e681: 8 total, 8 up, 8 in 2024-09-11T08:45:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:29 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:45:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:29 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:45:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:29 smithi172 ceph-mon[32987]: pgmap v1066: 204 pgs: 12 unknown, 192 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:45:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: osdmap e682: 8 total, 8 up, 8 in 2024-09-11T08:45:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:45:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:45:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:30.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:30.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:30.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:45:30.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[40592]: osdmap e683: 8 total, 8 up, 8 in 2024-09-11T08:45:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:45:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: osdmap e682: 8 total, 8 up, 8 in 2024-09-11T08:45:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:45:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:45:30.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:30.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:30.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:30.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:30.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:45:30.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:30 smithi043 ceph-mon[29355]: osdmap e683: 8 total, 8 up, 8 in 2024-09-11T08:45:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:45:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: osdmap e682: 8 total, 8 up, 8 in 2024-09-11T08:45:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:45:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:45:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:30.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:45:30.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:30 smithi172 ceph-mon[32987]: osdmap e683: 8 total, 8 up, 8 in 2024-09-11T08:45:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:31 smithi043 ceph-mon[40592]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:31.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:31 smithi043 ceph-mon[29355]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:31 smithi172 ceph-mon[32987]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:32.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:32.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:32.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]: dispatch 2024-09-11T08:45:32.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]: dispatch 2024-09-11T08:45:32.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:32.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:32.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]: dispatch 2024-09-11T08:45:32.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]: dispatch 2024-09-11T08:45:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:45:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]: dispatch 2024-09-11T08:45:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]: dispatch 2024-09-11T08:45:33.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:33 smithi043 ceph-mon[29355]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-11T08:45:33.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]': finished 2024-09-11T08:45:33.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:33 smithi043 ceph-mon[29355]: osdmap e684: 8 total, 8 up, 8 in 2024-09-11T08:45:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:33 smithi043 ceph-mon[40592]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-11T08:45:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]': finished 2024-09-11T08:45:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:33 smithi043 ceph-mon[40592]: osdmap e684: 8 total, 8 up, 8 in 2024-09-11T08:45:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:33 smithi172 ceph-mon[32987]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-11T08:45:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-138"}]': finished 2024-09-11T08:45:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:33 smithi172 ceph-mon[32987]: osdmap e684: 8 total, 8 up, 8 in 2024-09-11T08:45:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:45:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:33] "GET /metrics HTTP/1.1" 200 228771 "" "Prometheus/2.43.0" 2024-09-11T08:45:34.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:34 smithi043 ceph-mon[40592]: osdmap e685: 8 total, 8 up, 8 in 2024-09-11T08:45:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:34 smithi043 ceph-mon[29355]: osdmap e685: 8 total, 8 up, 8 in 2024-09-11T08:45:34.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:34 smithi172 ceph-mon[32987]: osdmap e685: 8 total, 8 up, 8 in 2024-09-11T08:45:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:35 smithi043 ceph-mon[40592]: pgmap v1073: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-11T08:45:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:35 smithi043 ceph-mon[40592]: osdmap e686: 8 total, 8 up, 8 in 2024-09-11T08:45:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:35.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:35 smithi043 ceph-mon[29355]: pgmap v1073: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-11T08:45:35.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:35 smithi043 ceph-mon[29355]: osdmap e686: 8 total, 8 up, 8 in 2024-09-11T08:45:35.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:35.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:45:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:35 smithi172 ceph-mon[32987]: pgmap v1073: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-11T08:45:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:35 smithi172 ceph-mon[32987]: osdmap e686: 8 total, 8 up, 8 in 2024-09-11T08:45:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:36 smithi043 ceph-mon[40592]: osdmap e687: 8 total, 8 up, 8 in 2024-09-11T08:45:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:36 smithi043 ceph-mon[29355]: osdmap e687: 8 total, 8 up, 8 in 2024-09-11T08:45:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:36.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:45:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:45:36.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:36.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:36 smithi172 ceph-mon[32987]: osdmap e687: 8 total, 8 up, 8 in 2024-09-11T08:45:36.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:36.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:45:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[40592]: pgmap v1076: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[40592]: osdmap e688: 8 total, 8 up, 8 in 2024-09-11T08:45:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:45:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:45:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[29355]: pgmap v1076: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[29355]: osdmap e688: 8 total, 8 up, 8 in 2024-09-11T08:45:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:45:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:45:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:37 smithi172 ceph-mon[32987]: pgmap v1076: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:45:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:37 smithi172 ceph-mon[32987]: osdmap e688: 8 total, 8 up, 8 in 2024-09-11T08:45:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:45:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:45:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:45:38.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[40592]: osdmap e689: 8 total, 8 up, 8 in 2024-09-11T08:45:38.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:45:38.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:45:38.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:45:38.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[40592]: osdmap e690: 8 total, 8 up, 8 in 2024-09-11T08:45:38.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:45:38.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[29355]: osdmap e689: 8 total, 8 up, 8 in 2024-09-11T08:45:38.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:45:38.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:45:38.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:45:38.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:38 smithi043 ceph-mon[29355]: osdmap e690: 8 total, 8 up, 8 in 2024-09-11T08:45:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:45:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:38 smithi172 ceph-mon[32987]: osdmap e689: 8 total, 8 up, 8 in 2024-09-11T08:45:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:45:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:45:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:45:38.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:38 smithi172 ceph-mon[32987]: osdmap e690: 8 total, 8 up, 8 in 2024-09-11T08:45:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[29355]: pgmap v1079: 204 pgs: 13 unknown, 191 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:45:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:45:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:45:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[29355]: osdmap e691: 8 total, 8 up, 8 in 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[40592]: pgmap v1079: 204 pgs: 13 unknown, 191 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[40592]: osdmap e691: 8 total, 8 up, 8 in 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:45:39.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:45:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:39 smithi172 ceph-mon[32987]: pgmap v1079: 204 pgs: 13 unknown, 191 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:45:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:45:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:45:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:39 smithi172 ceph-mon[32987]: osdmap e691: 8 total, 8 up, 8 in 2024-09-11T08:45:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:45:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:45:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:41 smithi043 ceph-mon[40592]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:41 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:45:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:41 smithi043 ceph-mon[40592]: osdmap e692: 8 total, 8 up, 8 in 2024-09-11T08:45:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:41 smithi043 ceph-mon[29355]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:41 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:45:41.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:41 smithi043 ceph-mon[29355]: osdmap e692: 8 total, 8 up, 8 in 2024-09-11T08:45:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:41 smithi172 ceph-mon[32987]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:41 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:45:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:41 smithi172 ceph-mon[32987]: osdmap e692: 8 total, 8 up, 8 in 2024-09-11T08:45:42.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:42.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:42.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:43 smithi043 ceph-mon[40592]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:45:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:43 smithi043 ceph-mon[29355]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:45:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:43 smithi172 ceph-mon[32987]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:45:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:45:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:43] "GET /metrics HTTP/1.1" 200 228781 "" "Prometheus/2.43.0" 2024-09-11T08:45:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[40592]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-11T08:45:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]: dispatch 2024-09-11T08:45:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]: dispatch 2024-09-11T08:45:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]: dispatch 2024-09-11T08:45:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]: dispatch 2024-09-11T08:45:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[29355]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-11T08:45:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]: dispatch 2024-09-11T08:45:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]: dispatch 2024-09-11T08:45:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]: dispatch 2024-09-11T08:45:45.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:45 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]: dispatch 2024-09-11T08:45:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:45:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:45 smithi172 ceph-mon[32987]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-11T08:45:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:45 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]: dispatch 2024-09-11T08:45:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:45 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]: dispatch 2024-09-11T08:45:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:45 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]: dispatch 2024-09-11T08:45:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:45 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]: dispatch 2024-09-11T08:45:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]': finished 2024-09-11T08:45:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:46 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]': finished 2024-09-11T08:45:46.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:46 smithi043 ceph-mon[29355]: osdmap e693: 8 total, 8 up, 8 in 2024-09-11T08:45:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]': finished 2024-09-11T08:45:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:46 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]': finished 2024-09-11T08:45:46.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:46 smithi043 ceph-mon[40592]: osdmap e693: 8 total, 8 up, 8 in 2024-09-11T08:45:46.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:45:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:45:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.4", "id": [5, 0]}]': finished 2024-09-11T08:45:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:46 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "256.6", "id": [3, 4]}]': finished 2024-09-11T08:45:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:46 smithi172 ceph-mon[32987]: osdmap e693: 8 total, 8 up, 8 in 2024-09-11T08:45:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:47 smithi043 ceph-mon[40592]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 735 B/s rd, 4.7 KiB/s wr, 1 op/s 2024-09-11T08:45:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:47 smithi043 ceph-mon[40592]: osdmap e694: 8 total, 8 up, 8 in 2024-09-11T08:45:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:47 smithi043 ceph-mon[29355]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 735 B/s rd, 4.7 KiB/s wr, 1 op/s 2024-09-11T08:45:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:47.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:47 smithi043 ceph-mon[29355]: osdmap e694: 8 total, 8 up, 8 in 2024-09-11T08:45:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:47 smithi172 ceph-mon[32987]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 735 B/s rd, 4.7 KiB/s wr, 1 op/s 2024-09-11T08:45:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:47 smithi172 ceph-mon[32987]: osdmap e694: 8 total, 8 up, 8 in 2024-09-11T08:45:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:49 smithi043 ceph-mon[40592]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 2 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:49 smithi043 ceph-mon[29355]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 2 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:49 smithi172 ceph-mon[32987]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 2 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:51 smithi043 ceph-mon[40592]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:51 smithi043 ceph-mon[29355]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:51.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:51 smithi172 ceph-mon[32987]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:53 smithi043 ceph-mon[40592]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 2.0 KiB/s wr, 0 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:53 smithi043 ceph-mon[29355]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 2.0 KiB/s wr, 0 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:53 smithi172 ceph-mon[32987]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 2.0 KiB/s wr, 0 op/s; 1.0 KiB/s, 0 objects/s recovering 2024-09-11T08:45:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:45:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:53] "GET /metrics HTTP/1.1" 200 228781 "" "Prometheus/2.43.0" 2024-09-11T08:45:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:55 smithi043 ceph-mon[40592]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.8 KiB/s wr, 1 op/s; 924 B/s, 0 objects/s recovering 2024-09-11T08:45:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:55 smithi043 ceph-mon[29355]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.8 KiB/s wr, 1 op/s; 924 B/s, 0 objects/s recovering 2024-09-11T08:45:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:55 smithi172 ceph-mon[32987]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.8 KiB/s wr, 1 op/s; 924 B/s, 0 objects/s recovering 2024-09-11T08:45:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:45:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:45:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:56.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:45:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:45:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:57 smithi043 ceph-mon[40592]: pgmap v1093: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.6 KiB/s wr, 1 op/s; 827 B/s, 0 objects/s recovering 2024-09-11T08:45:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:57.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:57 smithi043 ceph-mon[29355]: pgmap v1093: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.6 KiB/s wr, 1 op/s; 827 B/s, 0 objects/s recovering 2024-09-11T08:45:57.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:57 smithi172 ceph-mon[32987]: pgmap v1093: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.6 KiB/s wr, 1 op/s; 827 B/s, 0 objects/s recovering 2024-09-11T08:45:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:45:59 smithi043 ceph-mon[40592]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 1 op/s; 692 B/s, 0 objects/s recovering 2024-09-11T08:45:59.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:45:59 smithi043 ceph-mon[29355]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 1 op/s; 692 B/s, 0 objects/s recovering 2024-09-11T08:45:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:45:59 smithi172 ceph-mon[32987]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 1 op/s; 692 B/s, 0 objects/s recovering 2024-09-11T08:46:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:01 smithi043 ceph-mon[40592]: pgmap v1095: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:46:01.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:01 smithi043 ceph-mon[29355]: pgmap v1095: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:46:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:01 smithi172 ceph-mon[32987]: pgmap v1095: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:46:03.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:03 smithi043 ceph-mon[40592]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:46:03.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:03 smithi043 ceph-mon[29355]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:46:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:03 smithi172 ceph-mon[32987]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:46:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:46:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:03] "GET /metrics HTTP/1.1" 200 228786 "" "Prometheus/2.43.0" 2024-09-11T08:46:05.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:05 smithi043 ceph-mon[40592]: pgmap v1097: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-11T08:46:05.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:05 smithi043 ceph-mon[29355]: pgmap v1097: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-11T08:46:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:05 smithi172 ceph-mon[32987]: pgmap v1097: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-11T08:46:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:46:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]: dispatch 2024-09-11T08:46:06.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]: dispatch 2024-09-11T08:46:06.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:06.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]: dispatch 2024-09-11T08:46:06.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]: dispatch 2024-09-11T08:46:06.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:46:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:46:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]: dispatch 2024-09-11T08:46:06.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]: dispatch 2024-09-11T08:46:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:07 smithi043 ceph-mon[40592]: pgmap v1098: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-11T08:46:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]': finished 2024-09-11T08:46:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:07 smithi043 ceph-mon[40592]: osdmap e695: 8 total, 8 up, 8 in 2024-09-11T08:46:08.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:07 smithi043 ceph-mon[29355]: pgmap v1098: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-11T08:46:08.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]': finished 2024-09-11T08:46:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:07 smithi043 ceph-mon[29355]: osdmap e695: 8 total, 8 up, 8 in 2024-09-11T08:46:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:07 smithi172 ceph-mon[32987]: pgmap v1098: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-11T08:46:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-140"}]': finished 2024-09-11T08:46:08.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:07 smithi172 ceph-mon[32987]: osdmap e695: 8 total, 8 up, 8 in 2024-09-11T08:46:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[40592]: osdmap e696: 8 total, 8 up, 8 in 2024-09-11T08:46:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[40592]: pgmap v1101: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[40592]: osdmap e697: 8 total, 8 up, 8 in 2024-09-11T08:46:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[29355]: osdmap e696: 8 total, 8 up, 8 in 2024-09-11T08:46:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[29355]: pgmap v1101: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[29355]: osdmap e697: 8 total, 8 up, 8 in 2024-09-11T08:46:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:08 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:08 smithi172 ceph-mon[32987]: osdmap e696: 8 total, 8 up, 8 in 2024-09-11T08:46:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:08 smithi172 ceph-mon[32987]: pgmap v1101: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:08 smithi172 ceph-mon[32987]: osdmap e697: 8 total, 8 up, 8 in 2024-09-11T08:46:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:08 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:09.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:08 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:10.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:10.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:10 smithi043 ceph-mon[40592]: osdmap e698: 8 total, 8 up, 8 in 2024-09-11T08:46:10.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:10.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:10.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:10.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:10 smithi043 ceph-mon[29355]: osdmap e698: 8 total, 8 up, 8 in 2024-09-11T08:46:10.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:10.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:10 smithi172 ceph-mon[32987]: osdmap e698: 8 total, 8 up, 8 in 2024-09-11T08:46:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:10.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[40592]: pgmap v1104: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[40592]: osdmap e699: 8 total, 8 up, 8 in 2024-09-11T08:46:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:11.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:11.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[29355]: pgmap v1104: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[29355]: osdmap e699: 8 total, 8 up, 8 in 2024-09-11T08:46:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:11.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:11 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:11 smithi172 ceph-mon[32987]: pgmap v1104: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:11 smithi172 ceph-mon[32987]: osdmap e699: 8 total, 8 up, 8 in 2024-09-11T08:46:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:11 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:11 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]': finished 2024-09-11T08:46:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[40592]: osdmap e700: 8 total, 8 up, 8 in 2024-09-11T08:46:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]: dispatch 2024-09-11T08:46:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]: dispatch 2024-09-11T08:46:12.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]': finished 2024-09-11T08:46:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[29355]: osdmap e700: 8 total, 8 up, 8 in 2024-09-11T08:46:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]: dispatch 2024-09-11T08:46:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]: dispatch 2024-09-11T08:46:12.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-142"}]': finished 2024-09-11T08:46:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:12 smithi172 ceph-mon[32987]: osdmap e700: 8 total, 8 up, 8 in 2024-09-11T08:46:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]: dispatch 2024-09-11T08:46:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]: dispatch 2024-09-11T08:46:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[40592]: pgmap v1107: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:13.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]': finished 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[40592]: osdmap e701: 8 total, 8 up, 8 in 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[40592]: osdmap e702: 8 total, 8 up, 8 in 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[29355]: pgmap v1107: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]': finished 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[29355]: osdmap e701: 8 total, 8 up, 8 in 2024-09-11T08:46:13.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:46:13.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-mon[29355]: osdmap e702: 8 total, 8 up, 8 in 2024-09-11T08:46:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:13 smithi172 ceph-mon[32987]: pgmap v1107: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:13 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-142", "mode": "writeback"}]': finished 2024-09-11T08:46:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:13 smithi172 ceph-mon[32987]: osdmap e701: 8 total, 8 up, 8 in 2024-09-11T08:46:13.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:13.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:13.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:46:13.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:13 smithi172 ceph-mon[32987]: osdmap e702: 8 total, 8 up, 8 in 2024-09-11T08:46:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:46:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:13] "GET /metrics HTTP/1.1" 200 228721 "" "Prometheus/2.43.0" 2024-09-11T08:46:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:14.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:14 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:46:14.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:14 smithi172 ceph-mon[32987]: osdmap e703: 8 total, 8 up, 8 in 2024-09-11T08:46:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:14 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:46:14.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:14 smithi043 ceph-mon[40592]: osdmap e703: 8 total, 8 up, 8 in 2024-09-11T08:46:14.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:14 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:46:14.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:14 smithi043 ceph-mon[29355]: osdmap e703: 8 total, 8 up, 8 in 2024-09-11T08:46:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-mon[32987]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:46:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:46:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-mon[32987]: osdmap e704: 8 total, 8 up, 8 in 2024-09-11T08:46:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:15.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[40592]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:46:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[40592]: osdmap e704: 8 total, 8 up, 8 in 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[29355]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[29355]: osdmap e704: 8 total, 8 up, 8 in 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:15.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:16.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:46:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:46:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:17 smithi172 ceph-mon[32987]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:46:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:46:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:17 smithi172 ceph-mon[32987]: osdmap e705: 8 total, 8 up, 8 in 2024-09-11T08:46:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:17 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:46:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:17 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:46:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[40592]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:46:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:46:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[40592]: osdmap e705: 8 total, 8 up, 8 in 2024-09-11T08:46:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:46:17.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:46:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[29355]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:46:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:46:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[29355]: osdmap e705: 8 total, 8 up, 8 in 2024-09-11T08:46:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:46:17.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:17 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:46:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:46:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:18 smithi172 ceph-mon[32987]: osdmap e706: 8 total, 8 up, 8 in 2024-09-11T08:46:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:46:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:46:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:46:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:18 smithi043 ceph-mon[40592]: osdmap e706: 8 total, 8 up, 8 in 2024-09-11T08:46:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:46:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:46:18.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:46:18.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:18 smithi043 ceph-mon[29355]: osdmap e706: 8 total, 8 up, 8 in 2024-09-11T08:46:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:46:18.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:46:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:19 smithi172 ceph-mon[32987]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:46:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:19 smithi172 ceph-mon[32987]: osdmap e707: 8 total, 8 up, 8 in 2024-09-11T08:46:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:19 smithi043 ceph-mon[40592]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:46:19.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:19 smithi043 ceph-mon[40592]: osdmap e707: 8 total, 8 up, 8 in 2024-09-11T08:46:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:19 smithi043 ceph-mon[29355]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:46:19.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:19 smithi043 ceph-mon[29355]: osdmap e707: 8 total, 8 up, 8 in 2024-09-11T08:46:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:20.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:20.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:20.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:20.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:20.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:21 smithi172 ceph-mon[32987]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:46:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:21 smithi172 ceph-mon[32987]: osdmap e708: 8 total, 8 up, 8 in 2024-09-11T08:46:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[40592]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:46:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[40592]: osdmap e708: 8 total, 8 up, 8 in 2024-09-11T08:46:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:21.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[29355]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:46:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[29355]: osdmap e708: 8 total, 8 up, 8 in 2024-09-11T08:46:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:21.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]': finished 2024-09-11T08:46:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:22 smithi172 ceph-mon[32987]: osdmap e709: 8 total, 8 up, 8 in 2024-09-11T08:46:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:22.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]': finished 2024-09-11T08:46:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[40592]: osdmap e709: 8 total, 8 up, 8 in 2024-09-11T08:46:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:22.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]': finished 2024-09-11T08:46:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[29355]: osdmap e709: 8 total, 8 up, 8 in 2024-09-11T08:46:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-142"}]: dispatch 2024-09-11T08:46:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:23 smithi172 ceph-mon[32987]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:23 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:23 smithi172 ceph-mon[32987]: osdmap e710: 8 total, 8 up, 8 in 2024-09-11T08:46:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:23 smithi043 ceph-mon[40592]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:23 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:23 smithi043 ceph-mon[40592]: osdmap e710: 8 total, 8 up, 8 in 2024-09-11T08:46:23.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:23 smithi043 ceph-mon[29355]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:23.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:23 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:23.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:23 smithi043 ceph-mon[29355]: osdmap e710: 8 total, 8 up, 8 in 2024-09-11T08:46:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:46:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:23] "GET /metrics HTTP/1.1" 200 228721 "" "Prometheus/2.43.0" 2024-09-11T08:46:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:24 smithi172 ceph-mon[32987]: osdmap e711: 8 total, 8 up, 8 in 2024-09-11T08:46:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:24.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:24 smithi043 ceph-mon[40592]: osdmap e711: 8 total, 8 up, 8 in 2024-09-11T08:46:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:24.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:24.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:24 smithi043 ceph-mon[29355]: osdmap e711: 8 total, 8 up, 8 in 2024-09-11T08:46:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:24.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:46:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:25 smithi172 ceph-mon[32987]: pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:46:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:25 smithi172 ceph-mon[32987]: osdmap e712: 8 total, 8 up, 8 in 2024-09-11T08:46:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:25 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[40592]: pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:46:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[40592]: osdmap e712: 8 total, 8 up, 8 in 2024-09-11T08:46:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[29355]: pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:46:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[29355]: osdmap e712: 8 total, 8 up, 8 in 2024-09-11T08:46:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:25.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:26.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:46:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:46:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:26 smithi172 ceph-mon[32987]: osdmap e713: 8 total, 8 up, 8 in 2024-09-11T08:46:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:26 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:26.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:26 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]': finished 2024-09-11T08:46:26.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:26 smithi172 ceph-mon[32987]: osdmap e714: 8 total, 8 up, 8 in 2024-09-11T08:46:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[40592]: osdmap e713: 8 total, 8 up, 8 in 2024-09-11T08:46:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]': finished 2024-09-11T08:46:26.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[40592]: osdmap e714: 8 total, 8 up, 8 in 2024-09-11T08:46:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[29355]: osdmap e713: 8 total, 8 up, 8 in 2024-09-11T08:46:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-144"}]': finished 2024-09-11T08:46:26.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:26 smithi043 ceph-mon[29355]: osdmap e714: 8 total, 8 up, 8 in 2024-09-11T08:46:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:27 smithi172 ceph-mon[32987]: pgmap v1127: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:46:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:27 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:46:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:46:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:27 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:27.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:27 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]': finished 2024-09-11T08:46:27.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:27 smithi172 ceph-mon[32987]: osdmap e715: 8 total, 8 up, 8 in 2024-09-11T08:46:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[40592]: pgmap v1127: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:46:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:46:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:46:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]': finished 2024-09-11T08:46:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[40592]: osdmap e715: 8 total, 8 up, 8 in 2024-09-11T08:46:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[29355]: pgmap v1127: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:46:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:46:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:46:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-144", "mode": "readproxy"}]': finished 2024-09-11T08:46:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:27 smithi043 ceph-mon[29355]: osdmap e715: 8 total, 8 up, 8 in 2024-09-11T08:46:28.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:28 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:28.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:28 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:28.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:28 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:29 smithi172 ceph-mon[32987]: pgmap v1130: 204 pgs: 15 unknown, 189 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:46:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:29 smithi043 ceph-mon[40592]: pgmap v1130: 204 pgs: 15 unknown, 189 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:46:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:29 smithi043 ceph-mon[29355]: pgmap v1130: 204 pgs: 15 unknown, 189 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:46:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:30 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:30 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:30 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:30 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:30 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:30.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:30 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:31 smithi172 ceph-mon[32987]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 175 B/s wr, 1 op/s 2024-09-11T08:46:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:31 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:31 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:46:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:31 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:31 smithi043 ceph-mon[40592]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 175 B/s wr, 1 op/s 2024-09-11T08:46:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:31 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:31 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:46:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:31 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:31 smithi043 ceph-mon[29355]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 175 B/s wr, 1 op/s 2024-09-11T08:46:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:31 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:31 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:46:31.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:31 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:33 smithi172 ceph-mon[32987]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 897 B/s rd, 149 B/s wr, 1 op/s 2024-09-11T08:46:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:33 smithi043 ceph-mon[40592]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 897 B/s rd, 149 B/s wr, 1 op/s 2024-09-11T08:46:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:33 smithi043 ceph-mon[29355]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 897 B/s rd, 149 B/s wr, 1 op/s 2024-09-11T08:46:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:46:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:33] "GET /metrics HTTP/1.1" 200 228770 "" "Prometheus/2.43.0" 2024-09-11T08:46:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:35 smithi172 ceph-mon[32987]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-11T08:46:35.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:46:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:35 smithi043 ceph-mon[40592]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-11T08:46:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:35 smithi043 ceph-mon[29355]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-11T08:46:36.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:46:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:46:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:37 smithi172 ceph-mon[32987]: pgmap v1134: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-09-11T08:46:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:37 smithi043 ceph-mon[40592]: pgmap v1134: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-09-11T08:46:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:37 smithi043 ceph-mon[29355]: pgmap v1134: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-09-11T08:46:37.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:38.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:38.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:38.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:38.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[40592]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 94 B/s wr, 1 op/s 2024-09-11T08:46:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:46:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[40592]: osdmap e716: 8 total, 8 up, 8 in 2024-09-11T08:46:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:39.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[29355]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 94 B/s wr, 1 op/s 2024-09-11T08:46:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:46:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[29355]: osdmap e716: 8 total, 8 up, 8 in 2024-09-11T08:46:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:39.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:39 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:39 smithi172 ceph-mon[32987]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 94 B/s wr, 1 op/s 2024-09-11T08:46:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:46:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:39 smithi172 ceph-mon[32987]: osdmap e716: 8 total, 8 up, 8 in 2024-09-11T08:46:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:39.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:39 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]': finished 2024-09-11T08:46:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[40592]: osdmap e717: 8 total, 8 up, 8 in 2024-09-11T08:46:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:40.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]': finished 2024-09-11T08:46:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[29355]: osdmap e717: 8 total, 8 up, 8 in 2024-09-11T08:46:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:40.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:40 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:40 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]': finished 2024-09-11T08:46:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:40 smithi172 ceph-mon[32987]: osdmap e717: 8 total, 8 up, 8 in 2024-09-11T08:46:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:46:40.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:40.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:40 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-144"}]: dispatch 2024-09-11T08:46:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:41 smithi172 ceph-mon[32987]: pgmap v1138: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:41 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:41.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:41 smithi172 ceph-mon[32987]: osdmap e718: 8 total, 8 up, 8 in 2024-09-11T08:46:41.732 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:46:41 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=sqlstore.transactions t=2024-09-11T08:46:41.373612515Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:46:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:41 smithi043 ceph-mon[40592]: pgmap v1138: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:41 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:41 smithi043 ceph-mon[40592]: osdmap e718: 8 total, 8 up, 8 in 2024-09-11T08:46:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:41 smithi043 ceph-mon[29355]: pgmap v1138: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:46:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:41 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:41 smithi043 ceph-mon[29355]: osdmap e718: 8 total, 8 up, 8 in 2024-09-11T08:46:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[40592]: osdmap e719: 8 total, 8 up, 8 in 2024-09-11T08:46:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:42.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[40592]: osdmap e720: 8 total, 8 up, 8 in 2024-09-11T08:46:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[29355]: osdmap e719: 8 total, 8 up, 8 in 2024-09-11T08:46:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:42 smithi043 ceph-mon[29355]: osdmap e720: 8 total, 8 up, 8 in 2024-09-11T08:46:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:42 smithi172 ceph-mon[32987]: osdmap e719: 8 total, 8 up, 8 in 2024-09-11T08:46:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:46:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:46:42.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:42 smithi172 ceph-mon[32987]: osdmap e720: 8 total, 8 up, 8 in 2024-09-11T08:46:43.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:43] "GET /metrics HTTP/1.1" 200 228727 "" "Prometheus/2.43.0" 2024-09-11T08:46:43.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[40592]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:43.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[40592]: osdmap e721: 8 total, 8 up, 8 in 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[29355]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[29355]: osdmap e721: 8 total, 8 up, 8 in 2024-09-11T08:46:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:46:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:43 smithi172 ceph-mon[32987]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:46:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:46:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:43 smithi172 ceph-mon[32987]: osdmap e721: 8 total, 8 up, 8 in 2024-09-11T08:46:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:46:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:46:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]': finished 2024-09-11T08:46:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:44 smithi043 ceph-mon[40592]: osdmap e722: 8 total, 8 up, 8 in 2024-09-11T08:46:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:46:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]': finished 2024-09-11T08:46:44.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:44 smithi043 ceph-mon[29355]: osdmap e722: 8 total, 8 up, 8 in 2024-09-11T08:46:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:46:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-79312-111", "overlaypool": "test-rados-api-smithi043-79312-146"}]': finished 2024-09-11T08:46:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:44 smithi172 ceph-mon[32987]: osdmap e722: 8 total, 8 up, 8 in 2024-09-11T08:46:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:46:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:45 smithi172 ceph-mon[32987]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:46:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:45 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]: dispatch 2024-09-11T08:46:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]: dispatch 2024-09-11T08:46:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:45 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:45 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]': finished 2024-09-11T08:46:45.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:45 smithi172 ceph-mon[32987]: osdmap e723: 8 total, 8 up, 8 in 2024-09-11T08:46:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[40592]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:46:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]: dispatch 2024-09-11T08:46:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]: dispatch 2024-09-11T08:46:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]': finished 2024-09-11T08:46:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[40592]: osdmap e723: 8 total, 8 up, 8 in 2024-09-11T08:46:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[29355]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:46:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]: dispatch 2024-09-11T08:46:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]: dispatch 2024-09-11T08:46:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:46:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-79312-146", "mode": "writeback"}]': finished 2024-09-11T08:46:45.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:45 smithi043 ceph-mon[29355]: osdmap e723: 8 total, 8 up, 8 in 2024-09-11T08:46:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:46 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:46.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:46 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:46.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:46:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:46:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:46 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:46.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:46 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:46 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:46.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:46 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:46:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[40592]: pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:46:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:46:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[40592]: osdmap e724: 8 total, 8 up, 8 in 2024-09-11T08:46:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[29355]: pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:46:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:46:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[29355]: osdmap e724: 8 total, 8 up, 8 in 2024-09-11T08:46:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:47.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:47 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:47 smithi172 ceph-mon[32987]: pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:46:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:46:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:47 smithi172 ceph-mon[32987]: osdmap e724: 8 total, 8 up, 8 in 2024-09-11T08:46:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:47 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:47 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:46:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:46:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[40592]: osdmap e725: 8 total, 8 up, 8 in 2024-09-11T08:46:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:48.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:46:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[29355]: osdmap e725: 8 total, 8 up, 8 in 2024-09-11T08:46:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:48.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:48 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:46:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:48 smithi172 ceph-mon[32987]: osdmap e725: 8 total, 8 up, 8 in 2024-09-11T08:46:48.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:48 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:48 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:46:48.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:48 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:46:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[40592]: pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:46:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:46:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[40592]: osdmap e726: 8 total, 8 up, 8 in 2024-09-11T08:46:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[29355]: pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:46:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:46:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[29355]: osdmap e726: 8 total, 8 up, 8 in 2024-09-11T08:46:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:49 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:49 smithi172 ceph-mon[32987]: pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:46:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:49 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:46:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:46:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:49 smithi172 ceph-mon[32987]: osdmap e726: 8 total, 8 up, 8 in 2024-09-11T08:46:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:49 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:49 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:46:50.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:46:50.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:50 smithi043 ceph-mon[40592]: osdmap e727: 8 total, 8 up, 8 in 2024-09-11T08:46:50.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:50 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:46:50.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:50 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:46:50.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:46:50.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:50 smithi043 ceph-mon[29355]: osdmap e727: 8 total, 8 up, 8 in 2024-09-11T08:46:50.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:50 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:46:50.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:50 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:46:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:46:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:50 smithi172 ceph-mon[32987]: osdmap e727: 8 total, 8 up, 8 in 2024-09-11T08:46:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:50 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:46:50.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:50 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:46:51.882 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:51 smithi172 ceph-mon[32987]: pgmap v1153: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:46:51.882 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:51 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:46:51.883 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:51 smithi172 ceph-mon[32987]: osdmap e728: 8 total, 8 up, 8 in 2024-09-11T08:46:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:51 smithi043 ceph-mon[29355]: pgmap v1153: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:46:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:51 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:46:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:51 smithi043 ceph-mon[29355]: osdmap e728: 8 total, 8 up, 8 in 2024-09-11T08:46:51.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:51 smithi043 ceph-mon[40592]: pgmap v1153: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:46:51.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:51 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-146","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:46:51.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:51 smithi043 ceph-mon[40592]: osdmap e728: 8 total, 8 up, 8 in 2024-09-11T08:46:52.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:52 smithi043 ceph-mon[29355]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:46:52.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:52 smithi043 ceph-mon[40592]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:46:52.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:52 smithi172 ceph-mon[32987]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:46:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:53 smithi043 ceph-mon[40592]: pgmap v1155: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:46:53.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:53 smithi043 ceph-mon[29355]: pgmap v1155: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:46:53.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:46:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:53] "GET /metrics HTTP/1.1" 200 228727 "" "Prometheus/2.43.0" 2024-09-11T08:46:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:53 smithi172 ceph-mon[32987]: pgmap v1155: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:46:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:55 smithi172 ceph-mon[32987]: pgmap v1156: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:46:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:46:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:46:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:55 smithi043 ceph-mon[40592]: pgmap v1156: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:46:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:55 smithi043 ceph-mon[29355]: pgmap v1156: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:46:56.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:46:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:46:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:57 smithi043 ceph-mon[40592]: pgmap v1157: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 661 B/s rd, 0 B/s wr, 0 op/s 2024-09-11T08:46:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:57 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:57 smithi043 ceph-mon[29355]: pgmap v1157: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 661 B/s rd, 0 B/s wr, 0 op/s 2024-09-11T08:46:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:57 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:57.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:57 smithi172 ceph-mon[32987]: pgmap v1157: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 661 B/s rd, 0 B/s wr, 0 op/s 2024-09-11T08:46:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:57 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:46:59 smithi043 ceph-mon[40592]: pgmap v1158: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:46:59.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:46:59 smithi043 ceph-mon[29355]: pgmap v1158: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:46:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:46:59 smithi172 ceph-mon[32987]: pgmap v1158: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:47:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:00 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:00 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:01.008 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:00 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:01 smithi172 ceph-mon[32987]: pgmap v1159: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:47:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:01 smithi172 ceph-mon[32987]: osdmap e729: 8 total, 8 up, 8 in 2024-09-11T08:47:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:01 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[40592]: pgmap v1159: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:47:02.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[40592]: osdmap e729: 8 total, 8 up, 8 in 2024-09-11T08:47:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:02.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[29355]: pgmap v1159: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:47:02.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[29355]: osdmap e729: 8 total, 8 up, 8 in 2024-09-11T08:47:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:02.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:01 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]': finished 2024-09-11T08:47:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:02 smithi172 ceph-mon[32987]: osdmap e730: 8 total, 8 up, 8 in 2024-09-11T08:47:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:02 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:02.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:02 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]': finished 2024-09-11T08:47:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[29355]: osdmap e730: 8 total, 8 up, 8 in 2024-09-11T08:47:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]': finished 2024-09-11T08:47:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[40592]: osdmap e730: 8 total, 8 up, 8 in 2024-09-11T08:47:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:03.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:02 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-146"}]: dispatch 2024-09-11T08:47:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:03 smithi172 ceph-mon[32987]: pgmap v1162: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:47:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:03 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:03 smithi172 ceph-mon[32987]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:47:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:03 smithi172 ceph-mon[32987]: osdmap e731: 8 total, 8 up, 8 in 2024-09-11T08:47:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:03] "GET /metrics HTTP/1.1" 200 228802 "" "Prometheus/2.43.0" 2024-09-11T08:47:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-mon[40592]: pgmap v1162: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:47:04.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:04.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-mon[40592]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:47:04.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-mon[40592]: osdmap e731: 8 total, 8 up, 8 in 2024-09-11T08:47:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-mon[29355]: pgmap v1162: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:47:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-mon[29355]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:47:04.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:03 smithi043 ceph-mon[29355]: osdmap e731: 8 total, 8 up, 8 in 2024-09-11T08:47:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:04 smithi172 ceph-mon[32987]: osdmap e732: 8 total, 8 up, 8 in 2024-09-11T08:47:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:04 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:04.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:04 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:05.045 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:04 smithi043 ceph-mon[40592]: osdmap e732: 8 total, 8 up, 8 in 2024-09-11T08:47:05.045 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:04 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:05.045 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:04 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:05.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:04 smithi043 ceph-mon[29355]: osdmap e732: 8 total, 8 up, 8 in 2024-09-11T08:47:05.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:04 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:05.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:04 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:05.679 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:47:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:05 smithi172 ceph-mon[32987]: pgmap v1165: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:47:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:05 smithi172 ceph-mon[32987]: osdmap e733: 8 total, 8 up, 8 in 2024-09-11T08:47:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:05 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:47:05.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:05 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:47:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[40592]: pgmap v1165: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:47:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[40592]: osdmap e733: 8 total, 8 up, 8 in 2024-09-11T08:47:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:47:06.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:47:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[29355]: pgmap v1165: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:47:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[29355]: osdmap e733: 8 total, 8 up, 8 in 2024-09-11T08:47:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:47:06.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:05 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:47:06.686 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:47:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:47:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:47:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:06 smithi172 ceph-mon[32987]: osdmap e734: 8 total, 8 up, 8 in 2024-09-11T08:47:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:06 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:06.984 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:06 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:47:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:06 smithi043 ceph-mon[40592]: osdmap e734: 8 total, 8 up, 8 in 2024-09-11T08:47:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:06 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:07.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:06 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:07.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:47:07.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:06 smithi043 ceph-mon[29355]: osdmap e734: 8 total, 8 up, 8 in 2024-09-11T08:47:07.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:06 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:07.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:06 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:07 smithi172 ceph-mon[32987]: pgmap v1168: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:47:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]': finished 2024-09-11T08:47:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:07 smithi172 ceph-mon[32987]: osdmap e735: 8 total, 8 up, 8 in 2024-09-11T08:47:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:07 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:07 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[40592]: pgmap v1168: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:47:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]': finished 2024-09-11T08:47:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[40592]: osdmap e735: 8 total, 8 up, 8 in 2024-09-11T08:47:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:08.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[29355]: pgmap v1168: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]': finished 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[29355]: osdmap e735: 8 total, 8 up, 8 in 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:08.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:07 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-148"}]: dispatch 2024-09-11T08:47:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:08 smithi172 ceph-mon[32987]: osdmap e736: 8 total, 8 up, 8 in 2024-09-11T08:47:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:08 smithi172 ceph-mon[32987]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:08.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:08 smithi172 ceph-mon[32987]: pgmap v1171: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:08 smithi043 ceph-mon[40592]: osdmap e736: 8 total, 8 up, 8 in 2024-09-11T08:47:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:08 smithi043 ceph-mon[40592]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:09.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:08 smithi043 ceph-mon[40592]: pgmap v1171: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:08 smithi043 ceph-mon[29355]: osdmap e736: 8 total, 8 up, 8 in 2024-09-11T08:47:09.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:08 smithi043 ceph-mon[29355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:09.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:08 smithi043 ceph-mon[29355]: pgmap v1171: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:09 smithi172 ceph-mon[32987]: osdmap e737: 8 total, 8 up, 8 in 2024-09-11T08:47:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:09 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:09 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:09 smithi043 ceph-mon[40592]: osdmap e737: 8 total, 8 up, 8 in 2024-09-11T08:47:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:09 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:10.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:09 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:10.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:09 smithi043 ceph-mon[29355]: osdmap e737: 8 total, 8 up, 8 in 2024-09-11T08:47:10.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:09 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:10.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:09 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:10 smithi172 ceph-mon[32987]: osdmap e738: 8 total, 8 up, 8 in 2024-09-11T08:47:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:10 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-150"}]: dispatch 2024-09-11T08:47:10.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:10 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-150"}]: dispatch 2024-09-11T08:47:10.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:10 smithi172 ceph-mon[32987]: pgmap v1174: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:47:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[40592]: osdmap e738: 8 total, 8 up, 8 in 2024-09-11T08:47:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-150"}]: dispatch 2024-09-11T08:47:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-150"}]: dispatch 2024-09-11T08:47:11.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[40592]: pgmap v1174: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:47:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[29355]: osdmap e738: 8 total, 8 up, 8 in 2024-09-11T08:47:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-150"}]: dispatch 2024-09-11T08:47:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-150"}]: dispatch 2024-09-11T08:47:11.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:10 smithi043 ceph-mon[29355]: pgmap v1174: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:47:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:11 smithi172 ceph-mon[32987]: osdmap e739: 8 total, 8 up, 8 in 2024-09-11T08:47:12.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:11 smithi043 ceph-mon[40592]: osdmap e739: 8 total, 8 up, 8 in 2024-09-11T08:47:12.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:11 smithi043 ceph-mon[29355]: osdmap e739: 8 total, 8 up, 8 in 2024-09-11T08:47:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[40592]: osdmap e740: 8 total, 8 up, 8 in 2024-09-11T08:47:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[40592]: pgmap v1177: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-09-11T08:47:13.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[40592]: osdmap e741: 8 total, 8 up, 8 in 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[29355]: osdmap e740: 8 total, 8 up, 8 in 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[29355]: pgmap v1177: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:13.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:12 smithi043 ceph-mon[29355]: osdmap e741: 8 total, 8 up, 8 in 2024-09-11T08:47:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:12 smithi172 ceph-mon[32987]: osdmap e740: 8 total, 8 up, 8 in 2024-09-11T08:47:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:12 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:12 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:12 smithi172 ceph-mon[32987]: pgmap v1177: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-09-11T08:47:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:12 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:13.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:12 smithi172 ceph-mon[32987]: osdmap e741: 8 total, 8 up, 8 in 2024-09-11T08:47:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:13] "GET /metrics HTTP/1.1" 200 228773 "" "Prometheus/2.43.0" 2024-09-11T08:47:14.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:14.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:14.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-152"}]: dispatch 2024-09-11T08:47:14.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-152"}]: dispatch 2024-09-11T08:47:14.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[29355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-152"}]: dispatch 2024-09-11T08:47:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-152"}]: dispatch 2024-09-11T08:47:14.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:13 smithi043 ceph-mon[40592]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:13 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-152"}]: dispatch 2024-09-11T08:47:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:13 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-152"}]: dispatch 2024-09-11T08:47:14.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:13 smithi172 ceph-mon[32987]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:15.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:14 smithi043 ceph-mon[29355]: osdmap e742: 8 total, 8 up, 8 in 2024-09-11T08:47:15.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:14 smithi043 ceph-mon[29355]: pgmap v1180: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:47:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:14 smithi043 ceph-mon[40592]: osdmap e742: 8 total, 8 up, 8 in 2024-09-11T08:47:15.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:14 smithi043 ceph-mon[40592]: pgmap v1180: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:47:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:14 smithi172 ceph-mon[32987]: osdmap e742: 8 total, 8 up, 8 in 2024-09-11T08:47:15.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:14 smithi172 ceph-mon[32987]: pgmap v1180: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:47:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:47:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:15 smithi043 ceph-mon[40592]: osdmap e743: 8 total, 8 up, 8 in 2024-09-11T08:47:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:16.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:16.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:15 smithi043 ceph-mon[29355]: osdmap e743: 8 total, 8 up, 8 in 2024-09-11T08:47:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:16.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:16.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:15 smithi172 ceph-mon[32987]: osdmap e743: 8 total, 8 up, 8 in 2024-09-11T08:47:16.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:16.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:47:16.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:47:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: osdmap e744: 8 total, 8 up, 8 in 2024-09-11T08:47:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-111","var": "dedup_tier","val": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-111","var": "dedup_tier","val": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: pgmap v1183: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[40592]: osdmap e745: 8 total, 8 up, 8 in 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: osdmap e744: 8 total, 8 up, 8 in 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-111","var": "dedup_tier","val": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-111","var": "dedup_tier","val": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:17.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:17.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: pgmap v1183: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:47:17.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:17.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:16 smithi043 ceph-mon[29355]: osdmap e745: 8 total, 8 up, 8 in 2024-09-11T08:47:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-79312-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:47:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: osdmap e744: 8 total, 8 up, 8 in 2024-09-11T08:47:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-111","var": "dedup_tier","val": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-79312-111","var": "dedup_tier","val": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:17.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:17.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-79312-111", "tierpool": "test-rados-api-smithi043-79312-154"}]: dispatch 2024-09-11T08:47:17.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: pgmap v1183: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:47:17.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:17.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:16 smithi172 ceph-mon[32987]: osdmap e745: 8 total, 8 up, 8 in 2024-09-11T08:47:19.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:18 smithi043 ceph-mon[40592]: osdmap e746: 8 total, 8 up, 8 in 2024-09-11T08:47:19.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:18 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:19.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:18 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:19.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:18 smithi043 ceph-mon[40592]: pgmap v1186: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:18 smithi043 ceph-mon[29355]: osdmap e746: 8 total, 8 up, 8 in 2024-09-11T08:47:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:18 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:19.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:18 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:19.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:18 smithi043 ceph-mon[29355]: pgmap v1186: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:18 smithi172 ceph-mon[32987]: osdmap e746: 8 total, 8 up, 8 in 2024-09-11T08:47:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:18 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:18 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:19.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:18 smithi172 ceph-mon[32987]: pgmap v1186: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:19.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7861 ms) 2024-09-11T08:47:19.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-11T08:47:19.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12169 ms) 2024-09-11T08:47:19.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-11T08:47:19.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8051 ms) 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13086 ms) 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7704 ms) 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8053 ms) 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: ok, hit_set contains 255:602f83fe:::foo:head 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9058 ms) 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,0 2024-09-11T08:47:19.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first is 1726044341 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044344,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044344,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044347,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044347,1726044359,1726044360,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044347,1726044359,1726044360,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044347,1726044359,1726044360,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044347,1726044359,1726044360,1726044362,1726044363,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044347,1726044359,1726044360,1726044362,1726044363,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044341,1726044343,1726044347,1726044359,1726044360,1726044362,1726044363,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726044347,1726044359,1726044360,1726044362,1726044363,1726044365,1726044366,0 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first now 1726044347, trimmed 2024-09-11T08:47:19.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (34086 ms) 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: foo0 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (15066 ms) 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18064 ms) 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22161 ms) 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5034 ms) 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3030 ms) 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-11T08:47:19.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3021 ms) 2024-09-11T08:47:19.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-11T08:47:19.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3010 ms) 2024-09-11T08:47:19.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (245032 ms total) 2024-09-11T08:47:19.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-09-11T08:47:19.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-11T08:47:19.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (879877 ms total) 2024-09-11T08:47:19.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79065 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79065 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79288 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79288 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79757 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79757 2024-09-11T08:47:19.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79552 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79552 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79414 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79414 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79136 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79136 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79462 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79462 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79157 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79157 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79661 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79661 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79037 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79037 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79101 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79101 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79694 2024-09-11T08:47:19.781 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79694 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79028 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79028 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79049 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79049 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79328 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79328 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79440 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79440 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79378 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79378 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79728 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79728 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79644 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79644 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79248 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79248 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79575 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79575 2024-09-11T08:47:19.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79021 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79021 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79082 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79082 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79481 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79481 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79119 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79119 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79596 2024-09-11T08:47:19.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79596 2024-09-11T08:47:19.785 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:47:19.785 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=79616 2024-09-11T08:47:19.785 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 79616 2024-09-11T08:47:19.785 INFO:tasks.workunit.client.0.smithi043.stderr:+ exit 0 2024-09-11T08:47:19.785 INFO:tasks.workunit.client.0.smithi043.stderr:+ cleanup 2024-09-11T08:47:19.785 INFO:tasks.workunit.client.0.smithi043.stderr:+ pkill -P 79016 2024-09-11T08:47:19.793 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-09-11T08:47:19.794 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T08:47:19.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T08:47:19.826 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-11T08:47:19.827 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-11T08:47:19.827 DEBUG:teuthology.orchestra.run.smithi043:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d0e6828a2016d48cf25ad84064e50742bb1c39b9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-11T08:47:19.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ uuidgen 2024-09-11T08:47:19.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ p=992c53e8-268f-42a0-a098-268d5016ad85 2024-09-11T08:47:19.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool create 992c53e8-268f-42a0-a098-268d5016ad85 12 2024-09-11T08:47:19.985 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 -- 172.21.15.43:0/2346279595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 msgr2=0x7f9184108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:19.985 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 --2- 172.21.15.43:0/2346279595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 0x7f9184108420 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f917c009d00 tx=0x7f917c004d10 comp rx=0 tx=0).stop 2024-09-11T08:47:19.985 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 -- 172.21.15.43:0/2346279595 shutdown_connections 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 --2- 172.21.15.43:0/2346279595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 0x7f9184108420 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 --2- 172.21.15.43:0/2346279595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91840fb6a0 0x7f91840fbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 --2- 172.21.15.43:0/2346279595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9184101670 0x7f9184101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 -- 172.21.15.43:0/2346279595 >> 172.21.15.43:0/2346279595 conn(0x7f9184072570 msgr2=0x7f91840f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 -- 172.21.15.43:0/2346279595 shutdown_connections 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 -- 172.21.15.43:0/2346279595 wait complete. 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 Processor -- start 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.982+0000 7f918aae8640 1 -- start start 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f918aae8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91840fb6a0 0x7f9184112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f918aae8640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 0x7f9184112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f918aae8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9184101670 0x7f9184065ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:19.986 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f918aae8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91840662c0 con 0x7f9184101670 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f918aae8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184066430 con 0x7f91840fb6a0 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f918aae8640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91840665f0 con 0x7f91840fc050 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9183fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 0x7f9184112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9184101670 0x7f9184065ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9184101670 0x7f9184065ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46858/0 (socket says 172.21.15.43:46858) 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9183fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 0x7f9184112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:38316/0 (socket says 172.21.15.43:38316) 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 -- 172.21.15.43:0/3830152733 learned_addr learned my addr 172.21.15.43:0/3830152733 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9188884640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91840fb6a0 0x7f9184112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 -- 172.21.15.43:0/3830152733 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 msgr2=0x7f9184112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 0x7f9184112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 -- 172.21.15.43:0/3830152733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91840fb6a0 msgr2=0x7f9184112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91840fb6a0 0x7f9184112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:19.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 -- 172.21.15.43:0/3830152733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178005040 con 0x7f9184101670 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9183fff640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 0x7f9184112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f9189085640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9184101670 0x7f9184065ca0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f917c009cd0 tx=0x7f917c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.983+0000 7f91817fa640 1 -- 172.21.15.43:0/3830152733 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c038070 con 0x7f9184101670 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.984+0000 7f91817fa640 1 -- 172.21.15.43:0/3830152733 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f917c033e30 con 0x7f9184101670 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.984+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c0098b0 con 0x7f9184101670 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.984+0000 7f91817fa640 1 -- 172.21.15.43:0/3830152733 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c03ca70 con 0x7f9184101670 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.984+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841198a0 con 0x7f9184101670 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.985+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91840fcd90 con 0x7f9184101670 2024-09-11T08:47:19.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.985+0000 7f91817fa640 1 -- 172.21.15.43:0/3830152733 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f917c030070 con 0x7f9184101670 2024-09-11T08:47:19.989 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.986+0000 7f91817fa640 1 --2- 172.21.15.43:0/3830152733 >> v2:172.21.15.43:6800/588375333 conn(0x7f916c06ef80 0x7f916c071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:19.989 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.986+0000 7f91817fa640 1 -- 172.21.15.43:0/3830152733 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(748..748 src has 1..748) v4 ==== 7270+0+0 (secure 0 0 0) 0x7f917c0af200 con 0x7f9184101670 2024-09-11T08:47:19.989 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.986+0000 7f9188884640 1 --2- 172.21.15.43:0/3830152733 >> v2:172.21.15.43:6800/588375333 conn(0x7f916c06ef80 0x7f916c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:19.989 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.986+0000 7f9188884640 1 --2- 172.21.15.43:0/3830152733 >> v2:172.21.15.43:6800/588375333 conn(0x7f916c06ef80 0x7f916c071430 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7f9174007fd0 tx=0x7f9174009e90 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:19.990 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:19.990+0000 7f91817fa640 1 -- 172.21.15.43:0/3830152733 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f917c07bc90 con 0x7f9184101670 2024-09-11T08:47:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[40592]: osdmap e747: 8 total, 8 up, 8 in 2024-09-11T08:47:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:20.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:20.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[29355]: osdmap e747: 8 total, 8 up, 8 in 2024-09-11T08:47:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:20.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:19 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:20.155 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.155+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12} v 0) v1 -- 0x7f918410aa90 con 0x7f9184101670 2024-09-11T08:47:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:19 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:19 smithi172 ceph-mon[32987]: osdmap e747: 8 total, 8 up, 8 in 2024-09-11T08:47:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:19 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3258877446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:19 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]: dispatch 2024-09-11T08:47:20.773 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.773+0000 7f91817fa640 1 -- 172.21.15.43:0/3830152733 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]=0 pool '992c53e8-268f-42a0-a098-268d5016ad85' created v749) v1 ==== 176+0+0 (secure 0 0 0) 0x7f917c07b630 con 0x7f9184101670 2024-09-11T08:47:20.876 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.876+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12} v 0) v1 -- 0x7f918410e0e0 con 0x7f9184101670 2024-09-11T08:47:20.876 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.877+0000 7f91817fa640 1 -- 172.21.15.43:0/3830152733 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]=0 pool '992c53e8-268f-42a0-a098-268d5016ad85' already exists v749) v1 ==== 183+0+0 (secure 0 0 0) 0x7f917c07b810 con 0x7f9184101670 2024-09-11T08:47:20.877 INFO:tasks.workunit.client.0.smithi043.stderr:pool '992c53e8-268f-42a0-a098-268d5016ad85' already exists 2024-09-11T08:47:20.878 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 >> v2:172.21.15.43:6800/588375333 conn(0x7f916c06ef80 msgr2=0x7f916c071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 --2- 172.21.15.43:0/3830152733 >> v2:172.21.15.43:6800/588375333 conn(0x7f916c06ef80 0x7f916c071430 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7f9174007fd0 tx=0x7f9174009e90 comp rx=0 tx=0).stop 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9184101670 msgr2=0x7f9184065ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9184101670 0x7f9184065ca0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f917c009cd0 tx=0x7f917c004d10 comp rx=0 tx=0).stop 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 shutdown_connections 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 --2- 172.21.15.43:0/3830152733 >> v2:172.21.15.43:6800/588375333 conn(0x7f916c06ef80 0x7f916c071430 unknown :-1 s=CLOSED pgs=3460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f91840fc050 0x7f9184112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9184101670 0x7f9184065ca0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 --2- 172.21.15.43:0/3830152733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91840fb6a0 0x7f9184112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 >> 172.21.15.43:0/3830152733 conn(0x7f9184072570 msgr2=0x7f9184073210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 shutdown_connections 2024-09-11T08:47:20.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.879+0000 7f918aae8640 1 -- 172.21.15.43:0/3830152733 wait complete. 2024-09-11T08:47:20.887 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 992c53e8-268f-42a0-a098-268d5016ad85 max_objects 10 2024-09-11T08:47:20.969 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.967+0000 7f9373d1d640 1 -- 172.21.15.43:0/2766277881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fc740 msgr2=0x7f936c0fcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:20.969 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.967+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2766277881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fc740 0x7f936c0fcbb0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f93600048c0 tx=0x7f9360009da0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.969 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.967+0000 7f9373d1d640 1 -- 172.21.15.43:0/2766277881 shutdown_connections 2024-09-11T08:47:20.969 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.967+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2766277881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fc740 0x7f936c0fcbb0 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.969 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.967+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2766277881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f936c0fd0f0 0x7f936c109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.967+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2766277881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f936c102760 0x7f936c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.967+0000 7f9373d1d640 1 -- 172.21.15.43:0/2766277881 >> 172.21.15.43:0/2766277881 conn(0x7f936c0f82c0 msgr2=0x7f936c0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 -- 172.21.15.43:0/2766277881 shutdown_connections 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 -- 172.21.15.43:0/2766277881 wait complete. 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 Processor -- start 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 -- start start 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f936c0fc740 0x7f936c06f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fd0f0 0x7f936c06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f936c102760 0x7f936c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c11b320 con 0x7f936c102760 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c11b490 con 0x7f936c0fc740 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f9373d1d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c11b650 con 0x7f936c0fd0f0 2024-09-11T08:47:20.970 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f93722ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f936c102760 0x7f936c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f93712b8640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fd0f0 0x7f936c06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f93722ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f936c102760 0x7f936c0703c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46872/0 (socket says 172.21.15.43:46872) 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.968+0000 7f93722ba640 1 -- 172.21.15.43:0/2396630569 learned_addr learned my addr 172.21.15.43:0/2396630569 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f9371ab9640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f936c0fc740 0x7f936c06f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f93712b8640 1 -- 172.21.15.43:0/2396630569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f936c0fc740 msgr2=0x7f936c06f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f93712b8640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f936c0fc740 0x7f936c06f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f93712b8640 1 -- 172.21.15.43:0/2396630569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f936c102760 msgr2=0x7f936c0703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f93712b8640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f936c102760 0x7f936c0703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f93712b8640 1 -- 172.21.15.43:0/2396630569 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9364005040 con 0x7f936c0fd0f0 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f93722ba640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f936c102760 0x7f936c0703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f93712b8640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fd0f0 0x7f936c06fe80 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f9360004890 tx=0x7f9360009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f936a7fc640 1 -- 172.21.15.43:0/2396630569 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93600090d0 con 0x7f936c0fd0f0 2024-09-11T08:47:20.971 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f936a7fc640 1 -- 172.21.15.43:0/2396630569 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f9360009230 con 0x7f936c0fd0f0 2024-09-11T08:47:20.972 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f936a7fc640 1 -- 172.21.15.43:0/2396630569 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93600323d0 con 0x7f936c0fd0f0 2024-09-11T08:47:20.972 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9360005040 con 0x7f936c0fd0f0 2024-09-11T08:47:20.972 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.969+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c11bbb0 con 0x7f936c0fd0f0 2024-09-11T08:47:20.972 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.970+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c0fde80 con 0x7f936c0fd0f0 2024-09-11T08:47:20.973 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.970+0000 7f936a7fc640 1 -- 172.21.15.43:0/2396630569 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f9360030020 con 0x7f936c0fd0f0 2024-09-11T08:47:20.973 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.971+0000 7f936a7fc640 1 --2- 172.21.15.43:0/2396630569 >> v2:172.21.15.43:6800/588375333 conn(0x7f935406ef80 0x7f9354071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:20.973 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.971+0000 7f936a7fc640 1 -- 172.21.15.43:0/2396630569 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(749..749 src has 1..749) v4 ==== 7645+0+0 (secure 0 0 0) 0x7f93600adec0 con 0x7f936c0fd0f0 2024-09-11T08:47:20.973 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.971+0000 7f9371ab9640 1 --2- 172.21.15.43:0/2396630569 >> v2:172.21.15.43:6800/588375333 conn(0x7f935406ef80 0x7f9354071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:20.973 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.972+0000 7f9371ab9640 1 --2- 172.21.15.43:0/2396630569 >> v2:172.21.15.43:6800/588375333 conn(0x7f935406ef80 0x7f9354071430 secure :-1 s=READY pgs=3461 cs=0 l=1 rev1=1 crypto rx=0x7f9364007fd0 tx=0x7f9364009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:20.975 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:20.976+0000 7f936a7fc640 1 -- 172.21.15.43:0/2396630569 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f936007a970 con 0x7f936c0fd0f0 2024-09-11T08:47:21.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:20 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:21.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:20 smithi172 ceph-mon[32987]: osdmap e748: 8 total, 8 up, 8 in 2024-09-11T08:47:21.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:20 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]: dispatch 2024-09-11T08:47:21.133 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:20 smithi172 ceph-mon[32987]: pgmap v1189: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:21.140 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:21.141+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f936c070e30 con 0x7f936c0fd0f0 2024-09-11T08:47:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:20 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:20 smithi043 ceph-mon[40592]: osdmap e748: 8 total, 8 up, 8 in 2024-09-11T08:47:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:20 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]: dispatch 2024-09-11T08:47:21.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:20 smithi043 ceph-mon[40592]: pgmap v1189: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:20 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-79312-111"}]': finished 2024-09-11T08:47:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:20 smithi043 ceph-mon[29355]: osdmap e748: 8 total, 8 up, 8 in 2024-09-11T08:47:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:20 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]: dispatch 2024-09-11T08:47:21.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:20 smithi043 ceph-mon[29355]: pgmap v1189: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:21.776 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:21.776+0000 7f936a7fc640 1 -- 172.21.15.43:0/2396630569 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v750) v1 ==== 223+0+0 (secure 0 0 0) 0x7f93600400d0 con 0x7f936c0fd0f0 2024-09-11T08:47:21.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:21.885+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f936c10bba0 con 0x7f936c0fd0f0 2024-09-11T08:47:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]': finished 2024-09-11T08:47:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[40592]: osdmap e749: 8 total, 8 up, 8 in 2024-09-11T08:47:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]: dispatch 2024-09-11T08:47:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2396630569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:22.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]': finished 2024-09-11T08:47:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[29355]: osdmap e749: 8 total, 8 up, 8 in 2024-09-11T08:47:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]: dispatch 2024-09-11T08:47:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2396630569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:22.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:21 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]': finished 2024-09-11T08:47:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:21 smithi172 ceph-mon[32987]: osdmap e749: 8 total, 8 up, 8 in 2024-09-11T08:47:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3830152733' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pg_num": 12}]: dispatch 2024-09-11T08:47:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:21 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2396630569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:22.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:21 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:22.777 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.777+0000 7f936a7fc640 1 -- 172.21.15.43:0/2396630569 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v751) v1 ==== 223+0+0 (secure 0 0 0) 0x7f93600400d0 con 0x7f936c0fd0f0 2024-09-11T08:47:22.777 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 10 for pool 992c53e8-268f-42a0-a098-268d5016ad85 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 >> v2:172.21.15.43:6800/588375333 conn(0x7f935406ef80 msgr2=0x7f9354071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2396630569 >> v2:172.21.15.43:6800/588375333 conn(0x7f935406ef80 0x7f9354071430 secure :-1 s=READY pgs=3461 cs=0 l=1 rev1=1 crypto rx=0x7f9364007fd0 tx=0x7f9364009da0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fd0f0 msgr2=0x7f936c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fd0f0 0x7f936c06fe80 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f9360004890 tx=0x7f9360009da0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 shutdown_connections 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2396630569 >> v2:172.21.15.43:6800/588375333 conn(0x7f935406ef80 0x7f9354071430 unknown :-1 s=CLOSED pgs=3461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f936c0fd0f0 0x7f936c06fe80 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f936c102760 0x7f936c0703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 --2- 172.21.15.43:0/2396630569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f936c0fc740 0x7f936c06f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 >> 172.21.15.43:0/2396630569 conn(0x7f936c0f82c0 msgr2=0x7f936c100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:22.780 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 shutdown_connections 2024-09-11T08:47:22.781 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.780+0000 7f9373d1d640 1 -- 172.21.15.43:0/2396630569 wait complete. 2024-09-11T08:47:22.794 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool application enable 992c53e8-268f-42a0-a098-268d5016ad85 rados 2024-09-11T08:47:22.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2115117248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7280fd710 msgr2=0x7fa7280fdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:22.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2115117248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7280fd710 0x7fa7280fdae0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fa718004950 tx=0x7fa718009da0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2115117248 shutdown_connections 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2115117248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa728106cc0 0x7fa7281090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2115117248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7280fe0b0 0x7fa728106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2115117248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7280fd710 0x7fa7280fdae0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2115117248 >> 172.21.15.43:0/2115117248 conn(0x7fa728072530 msgr2=0x7fa7280f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2115117248 shutdown_connections 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2115117248 wait complete. 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 Processor -- start 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.880+0000 7fa72ca3f640 1 -- start start 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa72ca3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7280fd710 0x7fa728112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa72ca3f640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7280fe0b0 0x7fa728113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa72ca3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa728106cc0 0x7fa72810cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:22.882 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa72ca3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72810d510 con 0x7fa728106cc0 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa72ca3f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72810d680 con 0x7fa7280fd710 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa72ca3f640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72810d840 con 0x7fa7280fe0b0 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7280fe0b0 0x7fa728113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa726d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa728106cc0 0x7fa72810cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7280fe0b0 0x7fa728113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:60274/0 (socket says 172.21.15.43:60274) 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 -- 172.21.15.43:0/2981772533 learned_addr learned my addr 172.21.15.43:0/2981772533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 -- 172.21.15.43:0/2981772533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7280fd710 msgr2=0x7fa728112d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 --2- 172.21.15.43:0/2981772533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7280fd710 0x7fa728112d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 -- 172.21.15.43:0/2981772533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa728106cc0 msgr2=0x7fa72810cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 --2- 172.21.15.43:0/2981772533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa728106cc0 0x7fa72810cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 -- 172.21.15.43:0/2981772533 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200098b0 con 0x7fa7280fe0b0 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa726d9d640 1 --2- 172.21.15.43:0/2981772533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa728106cc0 0x7fa72810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:47:22.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71ffff640 1 --2- 172.21.15.43:0/2981772533 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7280fe0b0 0x7fa728113250 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b5e0 tx=0x7fa7100090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:22.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71effd640 1 -- 172.21.15.43:0/2981772533 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000f590 con 0x7fa7280fe0b0 2024-09-11T08:47:22.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71effd640 1 -- 172.21.15.43:0/2981772533 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fa71000fb60 con 0x7fa7280fe0b0 2024-09-11T08:47:22.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa718005040 con 0x7fa7280fe0b0 2024-09-11T08:47:22.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.881+0000 7fa71effd640 1 -- 172.21.15.43:0/2981772533 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710010a60 con 0x7fa7280fe0b0 2024-09-11T08:47:22.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.882+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa728119b10 con 0x7fa7280fe0b0 2024-09-11T08:47:22.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.882+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7280652f0 con 0x7fa7280fe0b0 2024-09-11T08:47:22.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.883+0000 7fa71effd640 1 -- 172.21.15.43:0/2981772533 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fa71000f6f0 con 0x7fa7280fe0b0 2024-09-11T08:47:22.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.884+0000 7fa71effd640 1 --2- 172.21.15.43:0/2981772533 >> v2:172.21.15.43:6800/588375333 conn(0x7fa71406ef80 0x7fa714071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:22.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.884+0000 7fa71effd640 1 -- 172.21.15.43:0/2981772533 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(751..751 src has 1..751) v4 ==== 7645+0+0 (secure 0 0 0) 0x7fa71008e7e0 con 0x7fa7280fe0b0 2024-09-11T08:47:22.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.884+0000 7fa72659c640 1 --2- 172.21.15.43:0/2981772533 >> v2:172.21.15.43:6800/588375333 conn(0x7fa71406ef80 0x7fa714071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:22.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.884+0000 7fa72659c640 1 --2- 172.21.15.43:0/2981772533 >> v2:172.21.15.43:6800/588375333 conn(0x7fa71406ef80 0x7fa714071430 secure :-1 s=READY pgs=3462 cs=0 l=1 rev1=1 crypto rx=0x7fa718004540 tx=0x7fa718035040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:22.888 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:22.888+0000 7fa71effd640 1 -- 172.21.15.43:0/2981772533 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa71005b060 con 0x7fa7280fe0b0 2024-09-11T08:47:23.055 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:23.055+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"} v 0) v1 -- 0x7fa72810e350 con 0x7fa7280fe0b0 2024-09-11T08:47:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:47:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[40592]: osdmap e750: 8 total, 8 up, 8 in 2024-09-11T08:47:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2396630569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[40592]: pgmap v1192: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:23.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:47:23.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[40592]: osdmap e751: 8 total, 8 up, 8 in 2024-09-11T08:47:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:47:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[29355]: osdmap e750: 8 total, 8 up, 8 in 2024-09-11T08:47:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2396630569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[29355]: pgmap v1192: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:47:23.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:22 smithi043 ceph-mon[29355]: osdmap e751: 8 total, 8 up, 8 in 2024-09-11T08:47:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:47:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:22 smithi172 ceph-mon[32987]: osdmap e750: 8 total, 8 up, 8 in 2024-09-11T08:47:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:22 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2396630569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:47:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:22 smithi172 ceph-mon[32987]: pgmap v1192: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:47:23.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:22 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:47:23.232 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:22 smithi172 ceph-mon[32987]: osdmap e751: 8 total, 8 up, 8 in 2024-09-11T08:47:23.797 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:23.797+0000 7fa71effd640 1 -- 172.21.15.43:0/2981772533 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]=0 enabled application 'rados' on pool '992c53e8-268f-42a0-a098-268d5016ad85' v752) v1 ==== 213+0+0 (secure 0 0 0) 0x7fa71005aa00 con 0x7fa7280fe0b0 2024-09-11T08:47:23.901 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:23.902+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"} v 0) v1 -- 0x7fa72810bbd0 con 0x7fa7280fe0b0 2024-09-11T08:47:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:47:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:23] "GET /metrics HTTP/1.1" 200 228773 "" "Prometheus/2.43.0" 2024-09-11T08:47:24.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:23 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2981772533' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:24.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:23 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:24.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:23 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2981772533' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:24.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:23 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:23 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2981772533' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:24.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:23 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:24.805 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.805+0000 7fa71effd640 1 -- 172.21.15.43:0/2981772533 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]=0 enabled application 'rados' on pool '992c53e8-268f-42a0-a098-268d5016ad85' v753) v1 ==== 213+0+0 (secure 0 0 0) 0x7fa71005abe0 con 0x7fa7280fe0b0 2024-09-11T08:47:24.805 INFO:tasks.workunit.client.0.smithi043.stderr:enabled application 'rados' on pool '992c53e8-268f-42a0-a098-268d5016ad85' 2024-09-11T08:47:24.808 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 >> v2:172.21.15.43:6800/588375333 conn(0x7fa71406ef80 msgr2=0x7fa714071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:24.808 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2981772533 >> v2:172.21.15.43:6800/588375333 conn(0x7fa71406ef80 0x7fa714071430 secure :-1 s=READY pgs=3462 cs=0 l=1 rev1=1 crypto rx=0x7fa718004540 tx=0x7fa718035040 comp rx=0 tx=0).stop 2024-09-11T08:47:24.808 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7280fe0b0 msgr2=0x7fa728113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:24.808 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2981772533 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7280fe0b0 0x7fa728113250 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b5e0 tx=0x7fa7100090f0 comp rx=0 tx=0).stop 2024-09-11T08:47:24.808 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 shutdown_connections 2024-09-11T08:47:24.808 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2981772533 >> v2:172.21.15.43:6800/588375333 conn(0x7fa71406ef80 0x7fa714071430 unknown :-1 s=CLOSED pgs=3462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:24.808 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2981772533 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7280fe0b0 0x7fa728113250 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:24.809 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2981772533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa728106cc0 0x7fa72810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:24.809 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 --2- 172.21.15.43:0/2981772533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7280fd710 0x7fa728112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:24.809 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 >> 172.21.15.43:0/2981772533 conn(0x7fa728072530 msgr2=0x7fa7280fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:24.809 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 shutdown_connections 2024-09-11T08:47:24.809 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:24.808+0000 7fa72ca3f640 1 -- 172.21.15.43:0/2981772533 wait complete. 2024-09-11T08:47:24.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ seq 1 10 2024-09-11T08:47:24.828 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:24.828 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj1 /etc/passwd 2024-09-11T08:47:24.864 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:24.864 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj2 /etc/passwd 2024-09-11T08:47:24.901 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:24.901 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj3 /etc/passwd 2024-09-11T08:47:24.935 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:24.935 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj4 /etc/passwd 2024-09-11T08:47:24.970 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:24.970 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj5 /etc/passwd 2024-09-11T08:47:25.008 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:25.008 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj6 /etc/passwd 2024-09-11T08:47:25.039 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:25.039 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj7 /etc/passwd 2024-09-11T08:47:25.071 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:25.071 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj8 /etc/passwd 2024-09-11T08:47:25.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:25.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj9 /etc/passwd 2024-09-11T08:47:25.135 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:47:25.135 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put obj10 /etc/passwd 2024-09-11T08:47:25.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]': finished 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[29355]: osdmap e752: 8 total, 8 up, 8 in 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2981772533' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[29355]: pgmap v1195: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]': finished 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[40592]: osdmap e752: 8 total, 8 up, 8 in 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2981772533' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:25.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:24 smithi043 ceph-mon[40592]: pgmap v1195: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:25.166 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-11T08:47:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]': finished 2024-09-11T08:47:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:24 smithi172 ceph-mon[32987]: osdmap e752: 8 total, 8 up, 8 in 2024-09-11T08:47:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:24 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2981772533' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:24 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]: dispatch 2024-09-11T08:47:25.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:24 smithi172 ceph-mon[32987]: pgmap v1195: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:47:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:26.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:25 smithi043 ceph-mon[40592]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:26.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:25 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]': finished 2024-09-11T08:47:26.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:25 smithi043 ceph-mon[40592]: osdmap e753: 8 total, 8 up, 8 in 2024-09-11T08:47:26.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:25 smithi043 ceph-mon[29355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:26.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:25 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]': finished 2024-09-11T08:47:26.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:25 smithi043 ceph-mon[29355]: osdmap e753: 8 total, 8 up, 8 in 2024-09-11T08:47:26.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:25 smithi172 ceph-mon[32987]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:47:26.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:25 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "app": "rados"}]': finished 2024-09-11T08:47:26.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:25 smithi172 ceph-mon[32987]: osdmap e753: 8 total, 8 up, 8 in 2024-09-11T08:47:26.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:47:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:47:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:26 smithi043 ceph-mon[40592]: pgmap v1197: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:47:27.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:26 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:27.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:26 smithi043 ceph-mon[29355]: pgmap v1197: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:47:27.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:26 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:26 smithi172 ceph-mon[32987]: pgmap v1197: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:47:27.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:26 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:28.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:28.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:28.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:29.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:28 smithi043 ceph-mon[40592]: pgmap v1198: 176 pgs: 176 active+clean; 645 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:47:29.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:28 smithi043 ceph-mon[29355]: pgmap v1198: 176 pgs: 176 active+clean; 645 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:47:29.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:28 smithi172 ceph-mon[32987]: pgmap v1198: 176 pgs: 176 active+clean; 645 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:47:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[40592]: pgmap v1199: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-11T08:47:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:47:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:47:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:47:31.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[29355]: pgmap v1199: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-11T08:47:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:47:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:47:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:47:31.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:31 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:31 smithi172 ceph-mon[32987]: pgmap v1199: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-11T08:47:31.770 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:31 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:47:31.770 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:31 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:47:31.770 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:31 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:47:31.770 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:31 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-mon[40592]: pgmap v1200: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 601 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-11T08:47:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-mon[40592]: pool '992c53e8-268f-42a0-a098-268d5016ad85' is full (reached quota's max_objects: 10) 2024-09-11T08:47:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-mon[40592]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:47:33.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-mon[40592]: osdmap e754: 8 total, 8 up, 8 in 2024-09-11T08:47:33.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-mon[29355]: pgmap v1200: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 601 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-11T08:47:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-mon[29355]: pool '992c53e8-268f-42a0-a098-268d5016ad85' is full (reached quota's max_objects: 10) 2024-09-11T08:47:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-mon[29355]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:47:33.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-mon[29355]: osdmap e754: 8 total, 8 up, 8 in 2024-09-11T08:47:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:33 smithi172 ceph-mon[32987]: pgmap v1200: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 601 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-11T08:47:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:33 smithi172 ceph-mon[32987]: pool '992c53e8-268f-42a0-a098-268d5016ad85' is full (reached quota's max_objects: 10) 2024-09-11T08:47:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:33 smithi172 ceph-mon[32987]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:47:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:33 smithi172 ceph-mon[32987]: osdmap e754: 8 total, 8 up, 8 in 2024-09-11T08:47:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:47:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:33] "GET /metrics HTTP/1.1" 200 226290 "" "Prometheus/2.43.0" 2024-09-11T08:47:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:35 smithi043 ceph-mon[40592]: pgmap v1202: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-11T08:47:35.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:35 smithi043 ceph-mon[29355]: pgmap v1202: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-11T08:47:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:35 smithi172 ceph-mon[32987]: pgmap v1202: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-11T08:47:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:47:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:36.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:47:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:47:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:37 smithi043 ceph-mon[40592]: pgmap v1203: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:47:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:37 smithi043 ceph-mon[29355]: pgmap v1203: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:47:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:37 smithi172 ceph-mon[32987]: pgmap v1203: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:47:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:39 smithi043 ceph-mon[29355]: pgmap v1204: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:47:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:39 smithi043 ceph-mon[40592]: pgmap v1204: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:47:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:39 smithi172 ceph-mon[32987]: pgmap v1204: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:47:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:41 smithi043 ceph-mon[40592]: pgmap v1205: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:47:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:41 smithi043 ceph-mon[29355]: pgmap v1205: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:47:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:41 smithi172 ceph-mon[32987]: pgmap v1205: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:47:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:43 smithi043 ceph-mon[40592]: pgmap v1206: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:47:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:43.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:43 smithi043 ceph-mon[29355]: pgmap v1206: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:47:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:43 smithi172 ceph-mon[32987]: pgmap v1206: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:47:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:47:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:43] "GET /metrics HTTP/1.1" 200 226294 "" "Prometheus/2.43.0" 2024-09-11T08:47:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:45 smithi043 ceph-mon[40592]: pgmap v1207: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:47:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:45 smithi043 ceph-mon[29355]: pgmap v1207: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:47:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:45 smithi172 ceph-mon[32987]: pgmap v1207: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:47:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:47:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:46.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:47:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:47:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:47 smithi043 ceph-mon[40592]: pgmap v1208: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:47 smithi043 ceph-mon[29355]: pgmap v1208: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:47 smithi172 ceph-mon[32987]: pgmap v1208: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:49 smithi043 ceph-mon[40592]: pgmap v1209: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:49 smithi043 ceph-mon[29355]: pgmap v1209: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:49.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:49 smithi172 ceph-mon[32987]: pgmap v1209: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:51 smithi043 ceph-mon[40592]: pgmap v1210: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:51 smithi043 ceph-mon[29355]: pgmap v1210: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:51.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:51 smithi172 ceph-mon[32987]: pgmap v1210: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:53 smithi043 ceph-mon[40592]: pgmap v1211: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:53 smithi043 ceph-mon[29355]: pgmap v1211: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:53 smithi172 ceph-mon[32987]: pgmap v1211: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:47:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:53] "GET /metrics HTTP/1.1" 200 226294 "" "Prometheus/2.43.0" 2024-09-11T08:47:55.168 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=148319 2024-09-11T08:47:55.168 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 992c53e8-268f-42a0-a098-268d5016ad85 max_objects 100 2024-09-11T08:47:55.169 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put onemore /etc/passwd 2024-09-11T08:47:55.263 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 -- 172.21.15.43:0/620475487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0248106cc0 msgr2=0x7f02481090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:55.263 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 --2- 172.21.15.43:0/620475487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0248106cc0 0x7f02481090a0 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f0238004890 tx=0x7f0238009da0 comp rx=0 tx=0).stop 2024-09-11T08:47:55.263 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 -- 172.21.15.43:0/620475487 shutdown_connections 2024-09-11T08:47:55.263 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 --2- 172.21.15.43:0/620475487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0248106cc0 0x7f02481090a0 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:55.263 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 --2- 172.21.15.43:0/620475487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02480fe0b0 0x7f0248106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:55.263 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 --2- 172.21.15.43:0/620475487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02480fd710 0x7f02480fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:55.263 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 -- 172.21.15.43:0/620475487 >> 172.21.15.43:0/620475487 conn(0x7f0248072530 msgr2=0x7f02480f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 -- 172.21.15.43:0/620475487 shutdown_connections 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.260+0000 7f024f034640 1 -- 172.21.15.43:0/620475487 wait complete. 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024f034640 1 Processor -- start 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024f034640 1 -- start start 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024f034640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02480fd710 0x7f0248112a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024f034640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02480fe0b0 0x7f0248112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024f034640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0248106cc0 0x7f024810cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024f034640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024810d200 con 0x7f0248106cc0 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024f034640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024810d370 con 0x7f02480fe0b0 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024f034640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024810d530 con 0x7f02480fd710 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024cdd0640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02480fd710 0x7f0248112a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:55.264 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024d5d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0248106cc0 0x7f024810cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024cdd0640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02480fd710 0x7f0248112a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:34646/0 (socket says 172.21.15.43:34646) 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.261+0000 7f024cdd0640 1 -- 172.21.15.43:0/3317500107 learned_addr learned my addr 172.21.15.43:0/3317500107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f0247fff640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02480fe0b0 0x7f0248112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024d5d1640 1 -- 172.21.15.43:0/3317500107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02480fd710 msgr2=0x7f0248112a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024d5d1640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02480fd710 0x7f0248112a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024d5d1640 1 -- 172.21.15.43:0/3317500107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02480fe0b0 msgr2=0x7f0248112f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024d5d1640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02480fe0b0 0x7f0248112f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024d5d1640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02400098b0 con 0x7f0248106cc0 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024cdd0640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02480fd710 0x7f0248112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024d5d1640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0248106cc0 0x7f024810cbb0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f023802a3d0 tx=0x7f0238009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238007d00 con 0x7f0248106cc0 2024-09-11T08:47:55.265 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0238005040 con 0x7f0248106cc0 2024-09-11T08:47:55.266 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f02380092e0 con 0x7f0248106cc0 2024-09-11T08:47:55.266 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238033690 con 0x7f0248106cc0 2024-09-11T08:47:55.266 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.262+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0248119870 con 0x7f0248106cc0 2024-09-11T08:47:55.267 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.263+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02480652f0 con 0x7f0248106cc0 2024-09-11T08:47:55.267 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.264+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f0238030020 con 0x7f0248106cc0 2024-09-11T08:47:55.267 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.264+0000 7f02457fa640 1 --2- 172.21.15.43:0/3317500107 >> v2:172.21.15.43:6800/588375333 conn(0x7f023006ef80 0x7f0230071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:55.267 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.264+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(754..754 src has 1..754) v4 ==== 7658+0+0 (secure 0 0 0) 0x7f02380ae670 con 0x7f0248106cc0 2024-09-11T08:47:55.267 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.264+0000 7f024cdd0640 1 --2- 172.21.15.43:0/3317500107 >> v2:172.21.15.43:6800/588375333 conn(0x7f023006ef80 0x7f0230071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:55.267 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.265+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=755}) v3 -- 0x7f023007a180 con 0x7f0248106cc0 2024-09-11T08:47:55.267 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.265+0000 7f024cdd0640 1 --2- 172.21.15.43:0/3317500107 >> v2:172.21.15.43:6800/588375333 conn(0x7f023006ef80 0x7f0230071430 secure :-1 s=READY pgs=3474 cs=0 l=1 rev1=1 crypto rx=0x7f0240009fd0 tx=0x7f0240004d10 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:55.269 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.269+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f023807af80 con 0x7f0248106cc0 2024-09-11T08:47:55.435 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:55.434+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f024810e090 con 0x7f0248106cc0 2024-09-11T08:47:55.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:55 smithi043 ceph-mon[40592]: pgmap v1212: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:55 smithi043 ceph-mon[29355]: pgmap v1212: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:55 smithi172 ceph-mon[32987]: pgmap v1212: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:47:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:47:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:47:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:56.381 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:56.380+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 7 ==== osd_map(755..755 src has 1..755) v4 ==== 620+0+0 (secure 0 0 0) 0x7f023807acc0 con 0x7f0248106cc0 2024-09-11T08:47:56.381 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:56.380+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=756}) v3 -- 0x7f023007ad50 con 0x7f0248106cc0 2024-09-11T08:47:56.384 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:56.384+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v755) v1 ==== 225+0+0 (secure 0 0 0) 0x7f023807a920 con 0x7f0248106cc0 2024-09-11T08:47:56.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:56.486+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f024810bbd0 con 0x7f0248106cc0 2024-09-11T08:47:56.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:56 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:47:56.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:56 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:47:56.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:56 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:47:56.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:47:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:47:57.386 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.386+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 9 ==== osd_map(756..756 src has 1..756) v4 ==== 620+0+0 (secure 0 0 0) 0x7f023807b8b0 con 0x7f0248106cc0 2024-09-11T08:47:57.387 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.386+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=757}) v3 -- 0x7f023007b1e0 con 0x7f0248106cc0 2024-09-11T08:47:57.390 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.389+0000 7f02457fa640 1 -- 172.21.15.43:0/3317500107 <== mon.0 v2:172.21.15.43:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v756) v1 ==== 225+0+0 (secure 0 0 0) 0x7f023807ab00 con 0x7f0248106cc0 2024-09-11T08:47:57.391 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 100 for pool 992c53e8-268f-42a0-a098-268d5016ad85 2024-09-11T08:47:57.394 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.393+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 >> v2:172.21.15.43:6800/588375333 conn(0x7f023006ef80 msgr2=0x7f0230071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:57.394 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.393+0000 7f024f034640 1 --2- 172.21.15.43:0/3317500107 >> v2:172.21.15.43:6800/588375333 conn(0x7f023006ef80 0x7f0230071430 secure :-1 s=READY pgs=3474 cs=0 l=1 rev1=1 crypto rx=0x7f0240009fd0 tx=0x7f0240004d10 comp rx=0 tx=0).stop 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.393+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0248106cc0 msgr2=0x7f024810cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.393+0000 7f024f034640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0248106cc0 0x7f024810cbb0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f023802a3d0 tx=0x7f0238009da0 comp rx=0 tx=0).stop 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.394+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 shutdown_connections 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.394+0000 7f024f034640 1 --2- 172.21.15.43:0/3317500107 >> v2:172.21.15.43:6800/588375333 conn(0x7f023006ef80 0x7f0230071430 unknown :-1 s=CLOSED pgs=3474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.394+0000 7f024f034640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0248106cc0 0x7f024810cbb0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.394+0000 7f024f034640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02480fe0b0 0x7f0248112f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.394+0000 7f024f034640 1 --2- 172.21.15.43:0/3317500107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02480fd710 0x7f0248112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.394+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 >> 172.21.15.43:0/3317500107 conn(0x7f0248072530 msgr2=0x7f02480fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.394+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 shutdown_connections 2024-09-11T08:47:57.395 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:57.394+0000 7f024f034640 1 -- 172.21.15.43:0/3317500107 wait complete. 2024-09-11T08:47:57.406 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 148319 2024-09-11T08:47:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[40592]: pgmap v1213: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:47:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[40592]: osdmap e755: 8 total, 8 up, 8 in 2024-09-11T08:47:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:47:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[29355]: pgmap v1213: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:47:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[29355]: osdmap e755: 8 total, 8 up, 8 in 2024-09-11T08:47:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:47:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:57 smithi172 ceph-mon[32987]: pgmap v1213: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:47:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:47:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:57 smithi172 ceph-mon[32987]: osdmap e755: 8 total, 8 up, 8 in 2024-09-11T08:47:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:57 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:47:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:58.199 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 0 -ne 0 ']' 2024-09-11T08:47:58.199 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-09-11T08:47:58.200 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put twomore /etc/passwd 2024-09-11T08:47:58.235 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 992c53e8-268f-42a0-a098-268d5016ad85 max_bytes 100 2024-09-11T08:47:58.320 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 -- 172.21.15.43:0/2513050856 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 msgr2=0x7f19e41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:58.320 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/2513050856 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 0x7f19e41019c0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f19d4004950 tx=0x7f19d4009da0 comp rx=0 tx=0).stop 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 -- 172.21.15.43:0/2513050856 shutdown_connections 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/2513050856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40fbd30 0x7f19e41083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/2513050856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e4101f90 0x7f19e40fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/2513050856 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 0x7f19e41019c0 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 -- 172.21.15.43:0/2513050856 >> 172.21.15.43:0/2513050856 conn(0x7f19e4072530 msgr2=0x7f19e40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 -- 172.21.15.43:0/2513050856 shutdown_connections 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 -- 172.21.15.43:0/2513050856 wait complete. 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 Processor -- start 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.318+0000 7f19e9d1d640 1 -- start start 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e9d1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e40fbd30 0x7f19e410bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e9d1d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 0x7f19e410c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:58.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e9d1d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e4101f90 0x7f19e410c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e9d1d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e411b2b0 con 0x7f19e40fbd30 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e9d1d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e411b3f0 con 0x7f19e4101f90 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e9d1d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e411b530 con 0x7f19e41015f0 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19daffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 0x7f19e410c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e40fbd30 0x7f19e410bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e40fbd30 0x7f19e410bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33046/0 (socket says 172.21.15.43:33046) 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19daffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 0x7f19e410c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:34698/0 (socket says 172.21.15.43:34698) 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 -- 172.21.15.43:0/3116314019 learned_addr learned my addr 172.21.15.43:0/3116314019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e3fff640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e4101f90 0x7f19e410c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 -- 172.21.15.43:0/3116314019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 msgr2=0x7f19e410c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 0x7f19e410c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:58.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 -- 172.21.15.43:0/3116314019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e4101f90 msgr2=0x7f19e410c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e4101f90 0x7f19e410c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 -- 172.21.15.43:0/3116314019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4005040 con 0x7f19e40fbd30 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19daffd640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 0x7f19e410c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e3fff640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e4101f90 0x7f19e410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e37fe640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e40fbd30 0x7f19e410bba0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f19d40049b0 tx=0x7f19d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.319+0000 7f19e0ff9640 1 -- 172.21.15.43:0/3116314019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d402a960 con 0x7f19e40fbd30 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.320+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e411b7b0 con 0x7f19e40fbd30 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.320+0000 7f19e0ff9640 1 -- 172.21.15.43:0/3116314019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f19d40325e0 con 0x7f19e40fbd30 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.320+0000 7f19e0ff9640 1 -- 172.21.15.43:0/3116314019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d403c6b0 con 0x7f19e40fbd30 2024-09-11T08:47:58.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.320+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e411bd00 con 0x7f19e40fbd30 2024-09-11T08:47:58.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.321+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e40fcd70 con 0x7f19e40fbd30 2024-09-11T08:47:58.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.321+0000 7f19e0ff9640 1 -- 172.21.15.43:0/3116314019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f19d4030020 con 0x7f19e40fbd30 2024-09-11T08:47:58.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.322+0000 7f19e0ff9640 1 --2- 172.21.15.43:0/3116314019 >> v2:172.21.15.43:6800/588375333 conn(0x7f19d006ef80 0x7f19d0071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:47:58.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.322+0000 7f19e0ff9640 1 -- 172.21.15.43:0/3116314019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(757..757 src has 1..757) v4 ==== 7658+0+0 (secure 0 0 0) 0x7f19d40ae6a0 con 0x7f19e40fbd30 2024-09-11T08:47:58.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.322+0000 7f19daffd640 1 --2- 172.21.15.43:0/3116314019 >> v2:172.21.15.43:6800/588375333 conn(0x7f19d006ef80 0x7f19d0071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:47:58.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.322+0000 7f19daffd640 1 --2- 172.21.15.43:0/3116314019 >> v2:172.21.15.43:6800/588375333 conn(0x7f19d006ef80 0x7f19d0071430 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7f19cc004fd0 tx=0x7f19cc00b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:47:58.326 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.326+0000 7f19e0ff9640 1 -- 172.21.15.43:0/3116314019 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f19d407afb0 con 0x7f19e40fbd30 2024-09-11T08:47:58.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:58.492+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f19e410ab40 con 0x7f19e40fbd30 2024-09-11T08:47:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:47:58.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[40592]: osdmap e756: 8 total, 8 up, 8 in 2024-09-11T08:47:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[40592]: pool '992c53e8-268f-42a0-a098-268d5016ad85' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:47:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[40592]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:47:58.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[40592]: osdmap e757: 8 total, 8 up, 8 in 2024-09-11T08:47:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:47:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[29355]: osdmap e756: 8 total, 8 up, 8 in 2024-09-11T08:47:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[29355]: pool '992c53e8-268f-42a0-a098-268d5016ad85' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:47:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[29355]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:47:58.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:58 smithi043 ceph-mon[29355]: osdmap e757: 8 total, 8 up, 8 in 2024-09-11T08:47:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:58 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3317500107' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:47:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:58 smithi172 ceph-mon[32987]: osdmap e756: 8 total, 8 up, 8 in 2024-09-11T08:47:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:58 smithi172 ceph-mon[32987]: pool '992c53e8-268f-42a0-a098-268d5016ad85' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:47:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:58 smithi172 ceph-mon[32987]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:47:58.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:58 smithi172 ceph-mon[32987]: osdmap e757: 8 total, 8 up, 8 in 2024-09-11T08:47:59.398 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:59.398+0000 7f19e0ff9640 1 -- 172.21.15.43:0/3116314019 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v758) v1 ==== 221+0+0 (secure 0 0 0) 0x7f19d407a950 con 0x7f19e40fbd30 2024-09-11T08:47:59.501 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:47:59.501+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f19e4065020 con 0x7f19e40fbd30 2024-09-11T08:47:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:59 smithi043 ceph-mon[40592]: pgmap v1217: 176 pgs: 176 active+clean; 719 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:47:59.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:47:59 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:47:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:59 smithi043 ceph-mon[29355]: pgmap v1217: 176 pgs: 176 active+clean; 719 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:47:59.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:47:59 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:47:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:59 smithi172 ceph-mon[32987]: pgmap v1217: 176 pgs: 176 active+clean; 719 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:47:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:47:59 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:48:00.401 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.400+0000 7f19e0ff9640 1 -- 172.21.15.43:0/3116314019 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v759) v1 ==== 221+0+0 (secure 0 0 0) 0x7f19d4080070 con 0x7f19e40fbd30 2024-09-11T08:48:00.401 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 100 for pool 992c53e8-268f-42a0-a098-268d5016ad85 2024-09-11T08:48:00.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 >> v2:172.21.15.43:6800/588375333 conn(0x7f19d006ef80 msgr2=0x7f19d0071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:00.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/3116314019 >> v2:172.21.15.43:6800/588375333 conn(0x7f19d006ef80 0x7f19d0071430 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7f19cc004fd0 tx=0x7f19cc00b040 comp rx=0 tx=0).stop 2024-09-11T08:48:00.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e40fbd30 msgr2=0x7f19e410bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:00.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e40fbd30 0x7f19e410bba0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f19d40049b0 tx=0x7f19d4009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:00.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 shutdown_connections 2024-09-11T08:48:00.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/3116314019 >> v2:172.21.15.43:6800/588375333 conn(0x7f19d006ef80 0x7f19d0071430 unknown :-1 s=CLOSED pgs=3476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:00.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19e41015f0 0x7f19e410c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:00.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e4101f90 0x7f19e410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:00.406 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 --2- 172.21.15.43:0/3116314019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e40fbd30 0x7f19e410bba0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:00.406 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 >> 172.21.15.43:0/3116314019 conn(0x7f19e4072530 msgr2=0x7f19e4073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:00.406 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 shutdown_connections 2024-09-11T08:48:00.406 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:00.404+0000 7f19e9d1d640 1 -- 172.21.15.43:0/3116314019 wait complete. 2024-09-11T08:48:00.416 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-11T08:48:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:48:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:00 smithi172 ceph-mon[32987]: osdmap e758: 8 total, 8 up, 8 in 2024-09-11T08:48:00.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:00 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:48:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:48:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:00 smithi043 ceph-mon[40592]: osdmap e758: 8 total, 8 up, 8 in 2024-09-11T08:48:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:00 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:48:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:48:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:00 smithi043 ceph-mon[29355]: osdmap e758: 8 total, 8 up, 8 in 2024-09-11T08:48:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:00 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:48:01.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:01 smithi172 ceph-mon[32987]: pgmap v1219: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:48:01.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:01 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:48:01.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:01 smithi172 ceph-mon[32987]: osdmap e759: 8 total, 8 up, 8 in 2024-09-11T08:48:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:01 smithi043 ceph-mon[40592]: pgmap v1219: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:48:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:01 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:48:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:01 smithi043 ceph-mon[40592]: osdmap e759: 8 total, 8 up, 8 in 2024-09-11T08:48:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:01 smithi043 ceph-mon[29355]: pgmap v1219: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:48:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:01 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3116314019' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:48:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:01 smithi043 ceph-mon[29355]: osdmap e759: 8 total, 8 up, 8 in 2024-09-11T08:48:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:03 smithi172 ceph-mon[32987]: pgmap v1221: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-11T08:48:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:03 smithi172 ceph-mon[32987]: pool '992c53e8-268f-42a0-a098-268d5016ad85' is full (reached quota's max_bytes: 100 B) 2024-09-11T08:48:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:03 smithi172 ceph-mon[32987]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:48:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:03 smithi172 ceph-mon[32987]: osdmap e760: 8 total, 8 up, 8 in 2024-09-11T08:48:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-mon[40592]: pgmap v1221: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-11T08:48:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-mon[40592]: pool '992c53e8-268f-42a0-a098-268d5016ad85' is full (reached quota's max_bytes: 100 B) 2024-09-11T08:48:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-mon[40592]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:48:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-mon[40592]: osdmap e760: 8 total, 8 up, 8 in 2024-09-11T08:48:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-mon[29355]: pgmap v1221: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-11T08:48:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-mon[29355]: pool '992c53e8-268f-42a0-a098-268d5016ad85' is full (reached quota's max_bytes: 100 B) 2024-09-11T08:48:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-mon[29355]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:48:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-mon[29355]: osdmap e760: 8 total, 8 up, 8 in 2024-09-11T08:48:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:48:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:03] "GET /metrics HTTP/1.1" 200 226243 "" "Prometheus/2.43.0" 2024-09-11T08:48:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:05 smithi172 ceph-mon[32987]: pgmap v1223: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:48:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:48:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:05.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:05 smithi043 ceph-mon[40592]: pgmap v1223: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:48:05.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:05 smithi043 ceph-mon[29355]: pgmap v1223: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:48:06.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:48:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:48:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:07 smithi172 ceph-mon[32987]: pgmap v1224: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 740 B/s rd, 0 op/s 2024-09-11T08:48:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:07.902 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:07 smithi043 ceph-mon[40592]: pgmap v1224: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 740 B/s rd, 0 op/s 2024-09-11T08:48:07.902 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:07.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:07 smithi043 ceph-mon[29355]: pgmap v1224: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 740 B/s rd, 0 op/s 2024-09-11T08:48:07.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:09 smithi172 ceph-mon[32987]: pgmap v1225: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:09 smithi043 ceph-mon[40592]: pgmap v1225: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:09 smithi043 ceph-mon[29355]: pgmap v1225: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:11 smithi172 ceph-mon[32987]: pgmap v1226: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:48:11.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:11 smithi043 ceph-mon[40592]: pgmap v1226: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:48:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:11 smithi043 ceph-mon[29355]: pgmap v1226: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:48:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:13 smithi172 ceph-mon[32987]: pgmap v1227: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:13 smithi043 ceph-mon[40592]: pgmap v1227: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:13 smithi043 ceph-mon[29355]: pgmap v1227: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:48:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:13] "GET /metrics HTTP/1.1" 200 226283 "" "Prometheus/2.43.0" 2024-09-11T08:48:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:15 smithi172 ceph-mon[32987]: pgmap v1228: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:48:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:48:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:15 smithi043 ceph-mon[40592]: pgmap v1228: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:48:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:15 smithi043 ceph-mon[29355]: pgmap v1228: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:48:16.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:48:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:48:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:17 smithi172 ceph-mon[32987]: pgmap v1229: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:17 smithi043 ceph-mon[40592]: pgmap v1229: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:17 smithi043 ceph-mon[29355]: pgmap v1229: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:19 smithi172 ceph-mon[32987]: pgmap v1230: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:19 smithi043 ceph-mon[40592]: pgmap v1230: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:19.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:19 smithi043 ceph-mon[29355]: pgmap v1230: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:21 smithi172 ceph-mon[32987]: pgmap v1231: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:21 smithi043 ceph-mon[40592]: pgmap v1231: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:21.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:21 smithi043 ceph-mon[29355]: pgmap v1231: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:23 smithi172 ceph-mon[32987]: pgmap v1232: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:23 smithi043 ceph-mon[40592]: pgmap v1232: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:23 smithi043 ceph-mon[29355]: pgmap v1232: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:48:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:23] "GET /metrics HTTP/1.1" 200 226283 "" "Prometheus/2.43.0" 2024-09-11T08:48:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:25 smithi172 ceph-mon[32987]: pgmap v1233: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:48:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:25 smithi043 ceph-mon[40592]: pgmap v1233: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:25.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:25 smithi043 ceph-mon[29355]: pgmap v1233: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:26.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:48:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:48:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:27 smithi172 ceph-mon[32987]: pgmap v1234: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:48:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:27 smithi043 ceph-mon[40592]: pgmap v1234: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:48:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:27 smithi043 ceph-mon[29355]: pgmap v1234: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:48:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:29 smithi172 ceph-mon[32987]: pgmap v1235: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:29 smithi043 ceph-mon[40592]: pgmap v1235: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:29 smithi043 ceph-mon[29355]: pgmap v1235: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:30.418 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=148406 2024-09-11T08:48:30.418 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 992c53e8-268f-42a0-a098-268d5016ad85 max_bytes 0 2024-09-11T08:48:30.418 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put two /etc/passwd 2024-09-11T08:48:30.514 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.512+0000 7f157a80f640 1 -- 172.21.15.43:0/1478541682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15741026d0 msgr2=0x7f1574102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:30.514 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.512+0000 7f157a80f640 1 --2- 172.21.15.43:0/1478541682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15741026d0 0x7f1574102aa0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f15640048c0 tx=0x7f1564009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:30.514 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.512+0000 7f157a80f640 1 -- 172.21.15.43:0/1478541682 shutdown_connections 2024-09-11T08:48:30.514 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.512+0000 7f157a80f640 1 --2- 172.21.15.43:0/1478541682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15740fce70 0x7f1574109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.512+0000 7f157a80f640 1 --2- 172.21.15.43:0/1478541682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1574103070 0x7f15740fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.512+0000 7f157a80f640 1 --2- 172.21.15.43:0/1478541682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15741026d0 0x7f1574102aa0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.512+0000 7f157a80f640 1 -- 172.21.15.43:0/1478541682 >> 172.21.15.43:0/1478541682 conn(0x7f15740f82e0 msgr2=0x7f15740fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.513+0000 7f157a80f640 1 -- 172.21.15.43:0/1478541682 shutdown_connections 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.513+0000 7f157a80f640 1 -- 172.21.15.43:0/1478541682 wait complete. 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.513+0000 7f157a80f640 1 Processor -- start 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.513+0000 7f157a80f640 1 -- start start 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.513+0000 7f157a80f640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15740fce70 0x7f1574112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f157a80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15741026d0 0x7f1574113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f157a80f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1574103070 0x7f157410ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f157a80f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157410d450 con 0x7f15741026d0 2024-09-11T08:48:30.515 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f157a80f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157410d5c0 con 0x7f1574103070 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f157a80f640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157410d780 con 0x7f15740fce70 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15740fce70 0x7f1574112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f15737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15741026d0 0x7f1574113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15740fce70 0x7f1574112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:46246/0 (socket says 172.21.15.43:46246) 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 -- 172.21.15.43:0/2541605248 learned_addr learned my addr 172.21.15.43:0/2541605248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1578dac640 1 --2- 172.21.15.43:0/2541605248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1574103070 0x7f157410ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 -- 172.21.15.43:0/2541605248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1574103070 msgr2=0x7f157410ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 --2- 172.21.15.43:0/2541605248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1574103070 0x7f157410ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 -- 172.21.15.43:0/2541605248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15741026d0 msgr2=0x7f1574113130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 --2- 172.21.15.43:0/2541605248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15741026d0 0x7f1574113130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0098b0 con 0x7f15740fce70 2024-09-11T08:48:30.516 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1573fff640 1 --2- 172.21.15.43:0/2541605248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15740fce70 0x7f1574112bf0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f1564004890 tx=0x7f1564009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:30.517 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640090d0 con 0x7f15740fce70 2024-09-11T08:48:30.517 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f1564009230 con 0x7f15740fce70 2024-09-11T08:48:30.517 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1564005040 con 0x7f15740fce70 2024-09-11T08:48:30.517 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.514+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156403b650 con 0x7f15740fce70 2024-09-11T08:48:30.517 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.515+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1574119a40 con 0x7f15740fce70 2024-09-11T08:48:30.518 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.516+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15740fde80 con 0x7f15740fce70 2024-09-11T08:48:30.518 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.516+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f1564030020 con 0x7f15740fce70 2024-09-11T08:48:30.518 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.517+0000 7f1570ff9640 1 --2- 172.21.15.43:0/2541605248 >> v2:172.21.15.43:6800/588375333 conn(0x7f155c06ef80 0x7f155c071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:30.518 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.517+0000 7f15737fe640 1 --2- 172.21.15.43:0/2541605248 >> v2:172.21.15.43:6800/588375333 conn(0x7f155c06ef80 0x7f155c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:30.518 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.517+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(760..760 src has 1..760) v4 ==== 7658+0+0 (secure 0 0 0) 0x7f15640ae6b0 con 0x7f15740fce70 2024-09-11T08:48:30.518 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.517+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=761}) v3 -- 0x7f155c07a200 con 0x7f15740fce70 2024-09-11T08:48:30.518 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.517+0000 7f15737fe640 1 --2- 172.21.15.43:0/2541605248 >> v2:172.21.15.43:6800/588375333 conn(0x7f155c06ef80 0x7f155c071430 secure :-1 s=READY pgs=3478 cs=0 l=1 rev1=1 crypto rx=0x7f157410e4b0 tx=0x7f156800b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:30.521 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.521+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f156407b150 con 0x7f15740fce70 2024-09-11T08:48:30.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:30.687+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f157410bc00 con 0x7f15740fce70 2024-09-11T08:48:31.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:31.455+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(761..761 src has 1..761) v4 ==== 620+0+0 (secure 0 0 0) 0x7f156407ae90 con 0x7f15740fce70 2024-09-11T08:48:31.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:31.455+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=762}) v3 -- 0x7f155c07add0 con 0x7f15740fce70 2024-09-11T08:48:31.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:31.461+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v761) v1 ==== 217+0+0 (secure 0 0 0) 0x7f156407aaf0 con 0x7f15740fce70 2024-09-11T08:48:31.566 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:31.566+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f157404bf60 con 0x7f15740fce70 2024-09-11T08:48:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:31 smithi172 ceph-mon[32987]: pgmap v1236: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:31 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2541605248' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:31 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:31 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:48:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:31 smithi043 ceph-mon[40592]: pgmap v1236: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:31 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2541605248' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:31 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:31.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:31 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:48:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:31 smithi043 ceph-mon[29355]: pgmap v1236: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:31 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2541605248' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:31 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:31 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:48:32.462 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.462+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 9 ==== osd_map(762..762 src has 1..762) v4 ==== 620+0+0 (secure 0 0 0) 0x7f1564033e20 con 0x7f15740fce70 2024-09-11T08:48:32.462 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.462+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=763}) v3 -- 0x7f155c07b260 con 0x7f15740fce70 2024-09-11T08:48:32.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.467+0000 7f1570ff9640 1 -- 172.21.15.43:0/2541605248 <== mon.2 v2:172.21.15.43:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v762) v1 ==== 217+0+0 (secure 0 0 0) 0x7f1564085020 con 0x7f15740fce70 2024-09-11T08:48:32.467 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 >> v2:172.21.15.43:6800/588375333 conn(0x7f155c06ef80 msgr2=0x7f155c071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 --2- 172.21.15.43:0/2541605248 >> v2:172.21.15.43:6800/588375333 conn(0x7f155c06ef80 0x7f155c071430 secure :-1 s=READY pgs=3478 cs=0 l=1 rev1=1 crypto rx=0x7f157410e4b0 tx=0x7f156800b040 comp rx=0 tx=0).stop 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15740fce70 msgr2=0x7f1574112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 --2- 172.21.15.43:0/2541605248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15740fce70 0x7f1574112bf0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f1564004890 tx=0x7f1564009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 shutdown_connections 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 --2- 172.21.15.43:0/2541605248 >> v2:172.21.15.43:6800/588375333 conn(0x7f155c06ef80 0x7f155c071430 unknown :-1 s=CLOSED pgs=3478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 --2- 172.21.15.43:0/2541605248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1574103070 0x7f157410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 --2- 172.21.15.43:0/2541605248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15741026d0 0x7f1574113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 --2- 172.21.15.43:0/2541605248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15740fce70 0x7f1574112bf0 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 >> 172.21.15.43:0/2541605248 conn(0x7f15740f82e0 msgr2=0x7f1574100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:32.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 shutdown_connections 2024-09-11T08:48:32.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.470+0000 7f157a80f640 1 -- 172.21.15.43:0/2541605248 wait complete. 2024-09-11T08:48:32.481 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 992c53e8-268f-42a0-a098-268d5016ad85 max_objects 0 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.566+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2468172191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 msgr2=0x7f8d280fac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.566+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2468172191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 0x7f8d280fac30 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c004950 tx=0x7f8d1c009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2468172191 shutdown_connections 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2468172191 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d280fb360 0x7f8d28109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2468172191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 0x7f8d280fac30 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2468172191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d28102730 0x7f8d28102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2468172191 >> 172.21.15.43:0/2468172191 conn(0x7f8d28074ba0 msgr2=0x7f8d280f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2468172191 shutdown_connections 2024-09-11T08:48:32.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2468172191 wait complete. 2024-09-11T08:48:32.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 Processor -- start 2024-09-11T08:48:32.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.567+0000 7f8d2e75d640 1 -- start start 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d2e75d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d280fb360 0x7f8d28112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d2e75d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d28102730 0x7f8d28113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d2e75d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 0x7f8d2810cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d2e75d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2810d3c0 con 0x7f8d281030d0 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d2e75d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2810d530 con 0x7f8d28102730 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d2e75d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2810d6f0 con 0x7f8d280fb360 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d280fb360 0x7f8d28112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d2ccfa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 0x7f8d2810cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d280fb360 0x7f8d28112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:56038/0 (socket says 172.21.15.43:56038) 2024-09-11T08:48:32.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 -- 172.21.15.43:0/2795417152 learned_addr learned my addr 172.21.15.43:0/2795417152 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d277fe640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d28102730 0x7f8d28113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 -- 172.21.15.43:0/2795417152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d28102730 msgr2=0x7f8d28113180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d28102730 0x7f8d28113180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 -- 172.21.15.43:0/2795417152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 msgr2=0x7f8d2810cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 0x7f8d2810cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 -- 172.21.15.43:0/2795417152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200098b0 con 0x7f8d280fb360 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d2ccfa640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 0x7f8d2810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.568+0000 7f8d27fff640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d280fb360 0x7f8d28112c40 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f8d18002ad0 tx=0x7f8d18004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.569+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18007c30 con 0x7f8d280fb360 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.569+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8d1800f040 con 0x7f8d280fb360 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.569+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c005040 con 0x7f8d280fb360 2024-09-11T08:48:32.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.569+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18009320 con 0x7f8d280fb360 2024-09-11T08:48:32.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.569+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d28119a60 con 0x7f8d280fb360 2024-09-11T08:48:32.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.570+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d280fdf50 con 0x7f8d280fb360 2024-09-11T08:48:32.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.570+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f8d180096d0 con 0x7f8d280fb360 2024-09-11T08:48:32.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.571+0000 7f8d24ff9640 1 --2- 172.21.15.43:0/2795417152 >> v2:172.21.15.43:6800/588375333 conn(0x7f8d1006ef80 0x7f8d10071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:32.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.571+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(762..762 src has 1..762) v4 ==== 7658+0+0 (secure 0 0 0) 0x7f8d1808e910 con 0x7f8d280fb360 2024-09-11T08:48:32.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.571+0000 7f8d277fe640 1 --2- 172.21.15.43:0/2795417152 >> v2:172.21.15.43:6800/588375333 conn(0x7f8d1006ef80 0x7f8d10071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:32.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.571+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=763}) v3 -- 0x7f8d1007a180 con 0x7f8d280fb360 2024-09-11T08:48:32.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.571+0000 7f8d277fe640 1 --2- 172.21.15.43:0/2795417152 >> v2:172.21.15.43:6800/588375333 conn(0x7f8d1006ef80 0x7f8d10071430 secure :-1 s=READY pgs=3479 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009040 tx=0x7f8d1c009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:32.574 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.575+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8d1805b250 con 0x7f8d280fb360 2024-09-11T08:48:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:48:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:32 smithi172 ceph-mon[32987]: osdmap e761: 8 total, 8 up, 8 in 2024-09-11T08:48:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:32 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2541605248' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:32 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:32 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:48:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:32 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:48:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:32 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:32.742 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:32.742+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f8d2810bb60 con 0x7f8d280fb360 2024-09-11T08:48:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:48:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[40592]: osdmap e761: 8 total, 8 up, 8 in 2024-09-11T08:48:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2541605248' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:48:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:48:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:48:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[29355]: osdmap e761: 8 total, 8 up, 8 in 2024-09-11T08:48:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2541605248' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:48:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:48:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:48:32.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:32 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:33.195 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:33.195+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(763..763 src has 1..763) v4 ==== 620+0+0 (secure 0 0 0) 0x7f8d18007d90 con 0x7f8d280fb360 2024-09-11T08:48:33.195 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:33.195+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=764}) v3 -- 0x7f8d1007ace0 con 0x7f8d280fb360 2024-09-11T08:48:33.199 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:33.200+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v763) v1 ==== 221+0+0 (secure 0 0 0) 0x7f8d1805abf0 con 0x7f8d280fb360 2024-09-11T08:48:33.301 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:33.301+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f8d2804bf60 con 0x7f8d280fb360 2024-09-11T08:48:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: pgmap v1238: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:48:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: osdmap e762: 8 total, 8 up, 8 in 2024-09-11T08:48:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2795417152' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: pool '992c53e8-268f-42a0-a098-268d5016ad85' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:48:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:48:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:48:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: osdmap e763: 8 total, 8 up, 8 in 2024-09-11T08:48:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/2795417152' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:33 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: pgmap v1238: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:48:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: osdmap e762: 8 total, 8 up, 8 in 2024-09-11T08:48:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2795417152' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: pool '992c53e8-268f-42a0-a098-268d5016ad85' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: osdmap e763: 8 total, 8 up, 8 in 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/2795417152' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: pgmap v1238: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: osdmap e762: 8 total, 8 up, 8 in 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2795417152' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: pool '992c53e8-268f-42a0-a098-268d5016ad85' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:48:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:48:33.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:48:33.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: osdmap e763: 8 total, 8 up, 8 in 2024-09-11T08:48:33.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/2795417152' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:33.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:48:34.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:48:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:33] "GET /metrics HTTP/1.1" 200 226246 "" "Prometheus/2.43.0" 2024-09-11T08:48:34.197 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.197+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 9 ==== osd_map(764..764 src has 1..764) v4 ==== 620+0+0 (secure 0 0 0) 0x7f8d1805af90 con 0x7f8d280fb360 2024-09-11T08:48:34.203 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.203+0000 7f8d24ff9640 1 -- 172.21.15.43:0/2795417152 <== mon.2 v2:172.21.15.43:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v764) v1 ==== 221+0+0 (secure 0 0 0) 0x7f8d1805add0 con 0x7f8d280fb360 2024-09-11T08:48:34.203 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 2024-09-11T08:48:34.206 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 >> v2:172.21.15.43:6800/588375333 conn(0x7f8d1006ef80 msgr2=0x7f8d10071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:34.206 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2795417152 >> v2:172.21.15.43:6800/588375333 conn(0x7f8d1006ef80 0x7f8d10071430 secure :-1 s=READY pgs=3479 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009040 tx=0x7f8d1c009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.206 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d280fb360 msgr2=0x7f8d28112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:34.206 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d280fb360 0x7f8d28112c40 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f8d18002ad0 tx=0x7f8d18004600 comp rx=0 tx=0).stop 2024-09-11T08:48:34.206 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 shutdown_connections 2024-09-11T08:48:34.207 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2795417152 >> v2:172.21.15.43:6800/588375333 conn(0x7f8d1006ef80 0x7f8d10071430 unknown :-1 s=CLOSED pgs=3479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.207 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d281030d0 0x7f8d2810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.207 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d28102730 0x7f8d28113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.207 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 --2- 172.21.15.43:0/2795417152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d280fb360 0x7f8d28112c40 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.207 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 >> 172.21.15.43:0/2795417152 conn(0x7f8d28074ba0 msgr2=0x7f8d281002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:34.207 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 shutdown_connections 2024-09-11T08:48:34.208 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.206+0000 7f8d2e75d640 1 -- 172.21.15.43:0/2795417152 wait complete. 2024-09-11T08:48:34.217 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 148406 2024-09-11T08:48:34.217 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 0 -ne 0 ']' 2024-09-11T08:48:34.217 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-09-11T08:48:34.217 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put three /etc/passwd 2024-09-11T08:48:34.252 INFO:tasks.workunit.client.0.smithi043.stderr:++ uuidgen 2024-09-11T08:48:34.253 INFO:tasks.workunit.client.0.smithi043.stderr:+ pp=741af959-956f-4579-9aef-0637359b7771 2024-09-11T08:48:34.253 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool create 741af959-956f-4579-9aef-0637359b7771 12 2024-09-11T08:48:34.337 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.335+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/4019288283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 msgr2=0x7f9c04101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:34.337 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.335+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/4019288283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 0x7f9c04101a40 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f9c00004980 tx=0x7f9c00009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.337 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.335+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/4019288283 shutdown_connections 2024-09-11T08:48:34.337 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.335+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/4019288283 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c040fc050 0x7f9c04108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.335+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/4019288283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c040fb6a0 0x7f9c040fbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.335+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/4019288283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 0x7f9c04101a40 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.335+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/4019288283 >> 172.21.15.43:0/4019288283 conn(0x7f9c04072570 msgr2=0x7f9c040f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/4019288283 shutdown_connections 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/4019288283 wait complete. 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 Processor -- start 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 -- start start 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c040fb6a0 0x7f9c04112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c040fc050 0x7f9c04113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 0x7f9c04065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04066360 con 0x7f9c04101670 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040664d0 con 0x7f9c040fc050 2024-09-11T08:48:34.338 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0bb9d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04066690 con 0x7f9c040fb6a0 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0a13a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 0x7f9c04065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c09939640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c040fb6a0 0x7f9c04112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0a13a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 0x7f9c04065d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46422/0 (socket says 172.21.15.43:46422) 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c09939640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c040fb6a0 0x7f9c04112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:56088/0 (socket says 172.21.15.43:56088) 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c0a13a640 1 -- 172.21.15.43:0/3883979774 learned_addr learned my addr 172.21.15.43:0/3883979774 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.336+0000 7f9c09138640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c040fc050 0x7f9c04113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c0a13a640 1 -- 172.21.15.43:0/3883979774 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c040fb6a0 msgr2=0x7f9c04112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c0a13a640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c040fb6a0 0x7f9c04112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c0a13a640 1 -- 172.21.15.43:0/3883979774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c040fc050 msgr2=0x7f9c04113120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c0a13a640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c040fc050 0x7f9c04113120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c0a13a640 1 -- 172.21.15.43:0/3883979774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c00005040 con 0x7f9c04101670 2024-09-11T08:48:34.339 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c09939640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c040fb6a0 0x7f9c04112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:48:34.340 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c09138640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c040fc050 0x7f9c04113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:48:34.340 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c0a13a640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 0x7f9c04065d10 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00ece0 tx=0x7f9bfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:34.340 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9bfa7fc640 1 -- 172.21.15.43:0/3883979774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00ee40 con 0x7f9c04101670 2024-09-11T08:48:34.340 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9bfa7fc640 1 -- 172.21.15.43:0/3883979774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f9bfc00f040 con 0x7f9c04101670 2024-09-11T08:48:34.340 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c04066910 con 0x7f9c04101670 2024-09-11T08:48:34.340 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9bfa7fc640 1 -- 172.21.15.43:0/3883979774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc009420 con 0x7f9c04101670 2024-09-11T08:48:34.340 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.337+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c04119ba0 con 0x7f9c04101670 2024-09-11T08:48:34.341 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.338+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c040fcd90 con 0x7f9c04101670 2024-09-11T08:48:34.341 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.338+0000 7f9bfa7fc640 1 -- 172.21.15.43:0/3883979774 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f9bfc007890 con 0x7f9c04101670 2024-09-11T08:48:34.341 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.339+0000 7f9bfa7fc640 1 --2- 172.21.15.43:0/3883979774 >> v2:172.21.15.43:6800/588375333 conn(0x7f9bf006ef80 0x7f9bf0071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:34.341 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.339+0000 7f9c09939640 1 --2- 172.21.15.43:0/3883979774 >> v2:172.21.15.43:6800/588375333 conn(0x7f9bf006ef80 0x7f9bf0071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:34.341 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.339+0000 7f9bfa7fc640 1 -- 172.21.15.43:0/3883979774 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(764..764 src has 1..764) v4 ==== 7658+0+0 (secure 0 0 0) 0x7f9bfc08f200 con 0x7f9c04101670 2024-09-11T08:48:34.341 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.340+0000 7f9c09939640 1 --2- 172.21.15.43:0/3883979774 >> v2:172.21.15.43:6800/588375333 conn(0x7f9bf006ef80 0x7f9bf0071430 secure :-1 s=READY pgs=3481 cs=0 l=1 rev1=1 crypto rx=0x7f9c000094b0 tx=0x7f9c00009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:34.343 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.344+0000 7f9bfa7fc640 1 -- 172.21.15.43:0/3883979774 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f9bfc05b9f0 con 0x7f9c04101670 2024-09-11T08:48:34.509 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:34.510+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12} v 0) v1 -- 0x7f9c0410e220 con 0x7f9c04101670 2024-09-11T08:48:35.205 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.205+0000 7f9bfa7fc640 1 -- 172.21.15.43:0/3883979774 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]=0 pool '741af959-956f-4579-9aef-0637359b7771' created v765) v1 ==== 176+0+0 (secure 0 0 0) 0x7f9bfc05b390 con 0x7f9c04101670 2024-09-11T08:48:35.308 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.308+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12} v 0) v1 -- 0x7f9c04119d30 con 0x7f9c04101670 2024-09-11T08:48:35.308 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.308+0000 7f9bfa7fc640 1 -- 172.21.15.43:0/3883979774 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]=0 pool '741af959-956f-4579-9aef-0637359b7771' already exists v765) v1 ==== 183+0+0 (secure 0 0 0) 0x7f9bfc05b570 con 0x7f9c04101670 2024-09-11T08:48:35.308 INFO:tasks.workunit.client.0.smithi043.stderr:pool '741af959-956f-4579-9aef-0637359b7771' already exists 2024-09-11T08:48:35.310 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.310+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 >> v2:172.21.15.43:6800/588375333 conn(0x7f9bf006ef80 msgr2=0x7f9bf0071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.310+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/3883979774 >> v2:172.21.15.43:6800/588375333 conn(0x7f9bf006ef80 0x7f9bf0071430 secure :-1 s=READY pgs=3481 cs=0 l=1 rev1=1 crypto rx=0x7f9c000094b0 tx=0x7f9c00009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 msgr2=0x7f9c04065d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 0x7f9c04065d10 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00ece0 tx=0x7f9bfc00b040 comp rx=0 tx=0).stop 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 shutdown_connections 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/3883979774 >> v2:172.21.15.43:6800/588375333 conn(0x7f9bf006ef80 0x7f9bf0071430 unknown :-1 s=CLOSED pgs=3481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04101670 0x7f9c04065d10 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c040fc050 0x7f9c04113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 --2- 172.21.15.43:0/3883979774 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c040fb6a0 0x7f9c04112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 >> 172.21.15.43:0/3883979774 conn(0x7f9c04072570 msgr2=0x7f9c040ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 shutdown_connections 2024-09-11T08:48:35.311 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.311+0000 7f9c0bb9d640 1 -- 172.21.15.43:0/3883979774 wait complete. 2024-09-11T08:48:35.319 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool application enable 741af959-956f-4579-9aef-0637359b7771 rados 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 -- 172.21.15.43:0/1741099989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c103070 msgr2=0x7f485c0fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 --2- 172.21.15.43:0/1741099989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c103070 0x7f485c0fc740 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f4850004980 tx=0x7f4850009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 -- 172.21.15.43:0/1741099989 shutdown_connections 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 --2- 172.21.15.43:0/1741099989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c103070 0x7f485c0fc740 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 --2- 172.21.15.43:0/1741099989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485c0fce70 0x7f485c109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 --2- 172.21.15.43:0/1741099989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c1026d0 0x7f485c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 -- 172.21.15.43:0/1741099989 >> 172.21.15.43:0/1741099989 conn(0x7f485c0f82e0 msgr2=0x7f485c0fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 -- 172.21.15.43:0/1741099989 shutdown_connections 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 -- 172.21.15.43:0/1741099989 wait complete. 2024-09-11T08:48:35.402 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 Processor -- start 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.400+0000 7f486155d640 1 -- start start 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f486155d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c0fce70 0x7f485c112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f486155d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c1026d0 0x7f485c1130d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f486155d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485c103070 0x7f485c10cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f486155d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c10d3e0 con 0x7f485c1026d0 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f486155d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c10d550 con 0x7f485c103070 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f486155d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c10d710 con 0x7f485c0fce70 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c0fce70 0x7f485c112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c1026d0 0x7f485c1130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c0fce70 0x7f485c112b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:56098/0 (socket says 172.21.15.43:56098) 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 -- 172.21.15.43:0/1826338773 learned_addr learned my addr 172.21.15.43:0/1826338773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485b7fe640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485c103070 0x7f485c10cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:35.403 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 -- 172.21.15.43:0/1826338773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485c103070 msgr2=0x7f485c10cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485c103070 0x7f485c10cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 -- 172.21.15.43:0/1826338773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c1026d0 msgr2=0x7f485c1130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c1026d0 0x7f485c1130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 -- 172.21.15.43:0/1826338773 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c005040 con 0x7f485c0fce70 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485a7fc640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c1026d0 0x7f485c1130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f485affd640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c0fce70 0x7f485c112b90 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f484c009780 tx=0x7f484c009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f48437fe640 1 -- 172.21.15.43:0/1826338773 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c0099c0 con 0x7f485c0fce70 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4850005040 con 0x7f485c0fce70 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f48437fe640 1 -- 172.21.15.43:0/1826338773 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f484c00f040 con 0x7f485c0fce70 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f48437fe640 1 -- 172.21.15.43:0/1826338773 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c013ca0 con 0x7f485c0fce70 2024-09-11T08:48:35.404 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.401+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c119990 con 0x7f485c0fce70 2024-09-11T08:48:35.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.403+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c0fde80 con 0x7f485c0fce70 2024-09-11T08:48:35.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.403+0000 7f48437fe640 1 -- 172.21.15.43:0/1826338773 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f484c00ebb0 con 0x7f485c0fce70 2024-09-11T08:48:35.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.403+0000 7f48437fe640 1 --2- 172.21.15.43:0/1826338773 >> v2:172.21.15.43:6800/588375333 conn(0x7f484406ef80 0x7f4844071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:35.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.403+0000 7f48437fe640 1 -- 172.21.15.43:0/1826338773 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(765..765 src has 1..765) v4 ==== 8033+0+0 (secure 0 0 0) 0x7f484c08eb70 con 0x7f485c0fce70 2024-09-11T08:48:35.405 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.403+0000 7f485a7fc640 1 --2- 172.21.15.43:0/1826338773 >> v2:172.21.15.43:6800/588375333 conn(0x7f484406ef80 0x7f4844071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:35.406 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.404+0000 7f485a7fc640 1 --2- 172.21.15.43:0/1826338773 >> v2:172.21.15.43:6800/588375333 conn(0x7f484406ef80 0x7f4844071430 secure :-1 s=READY pgs=3482 cs=0 l=1 rev1=1 crypto rx=0x7f48500048a0 tx=0x7f4850009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:35.407 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.408+0000 7f48437fe640 1 -- 172.21.15.43:0/1826338773 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f484c05b330 con 0x7f485c0fce70 2024-09-11T08:48:35.481 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:48:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:35.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:35 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:48:35.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:35 smithi172 ceph-mon[32987]: osdmap e764: 8 total, 8 up, 8 in 2024-09-11T08:48:35.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:35 smithi172 ceph-mon[32987]: pgmap v1242: 176 pgs: 176 active+clean; 744 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:35.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:35 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]: dispatch 2024-09-11T08:48:35.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:35.572+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"} v 0) v1 -- 0x7f485c10bb60 con 0x7f485c0fce70 2024-09-11T08:48:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:35 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:48:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:35 smithi043 ceph-mon[40592]: osdmap e764: 8 total, 8 up, 8 in 2024-09-11T08:48:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:35 smithi043 ceph-mon[40592]: pgmap v1242: 176 pgs: 176 active+clean; 744 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:35.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:35 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]: dispatch 2024-09-11T08:48:35.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:35 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:48:35.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:35 smithi043 ceph-mon[29355]: osdmap e764: 8 total, 8 up, 8 in 2024-09-11T08:48:35.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:35 smithi043 ceph-mon[29355]: pgmap v1242: 176 pgs: 176 active+clean; 744 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:35.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:35 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]: dispatch 2024-09-11T08:48:36.207 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:36.208+0000 7f48437fe640 1 -- 172.21.15.43:0/1826338773 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]=0 enabled application 'rados' on pool '741af959-956f-4579-9aef-0637359b7771' v766) v1 ==== 213+0+0 (secure 0 0 0) 0x7f484c05acd0 con 0x7f485c0fce70 2024-09-11T08:48:36.315 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:36.316+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"} v 0) v1 -- 0x7f485c04bfe0 con 0x7f485c0fce70 2024-09-11T08:48:36.464 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]': finished 2024-09-11T08:48:36.464 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:36 smithi172 ceph-mon[32987]: osdmap e765: 8 total, 8 up, 8 in 2024-09-11T08:48:36.464 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]: dispatch 2024-09-11T08:48:36.464 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:36 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1826338773' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:36.464 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:36 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]': finished 2024-09-11T08:48:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[40592]: osdmap e765: 8 total, 8 up, 8 in 2024-09-11T08:48:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]: dispatch 2024-09-11T08:48:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1826338773' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]': finished 2024-09-11T08:48:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[29355]: osdmap e765: 8 total, 8 up, 8 in 2024-09-11T08:48:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3883979774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "741af959-956f-4579-9aef-0637359b7771", "pg_num": 12}]: dispatch 2024-09-11T08:48:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1826338773' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:36 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:36.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:48:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:48:37.219 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.219+0000 7f48437fe640 1 -- 172.21.15.43:0/1826338773 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]=0 enabled application 'rados' on pool '741af959-956f-4579-9aef-0637359b7771' v767) v1 ==== 213+0+0 (secure 0 0 0) 0x7f484c05aeb0 con 0x7f485c0fce70 2024-09-11T08:48:37.219 INFO:tasks.workunit.client.0.smithi043.stderr:enabled application 'rados' on pool '741af959-956f-4579-9aef-0637359b7771' 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 >> v2:172.21.15.43:6800/588375333 conn(0x7f484406ef80 msgr2=0x7f4844071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 --2- 172.21.15.43:0/1826338773 >> v2:172.21.15.43:6800/588375333 conn(0x7f484406ef80 0x7f4844071430 secure :-1 s=READY pgs=3482 cs=0 l=1 rev1=1 crypto rx=0x7f48500048a0 tx=0x7f4850009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c0fce70 msgr2=0x7f485c112b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c0fce70 0x7f485c112b90 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f484c009780 tx=0x7f484c009da0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 shutdown_connections 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 --2- 172.21.15.43:0/1826338773 >> v2:172.21.15.43:6800/588375333 conn(0x7f484406ef80 0x7f4844071430 unknown :-1 s=CLOSED pgs=3482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485c103070 0x7f485c10cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c1026d0 0x7f485c1130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.222 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 --2- 172.21.15.43:0/1826338773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f485c0fce70 0x7f485c112b90 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.223 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 >> 172.21.15.43:0/1826338773 conn(0x7f485c0f82e0 msgr2=0x7f485c100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:37.223 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 shutdown_connections 2024-09-11T08:48:37.223 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.222+0000 7f486155d640 1 -- 172.21.15.43:0/1826338773 wait complete. 2024-09-11T08:48:37.233 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 741af959-956f-4579-9aef-0637359b7771 max_objects 10 2024-09-11T08:48:37.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 -- 172.21.15.43:0/3333264208 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80f00fb660 msgr2=0x7f80f0109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:37.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3333264208 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80f00fb660 0x7f80f0109520 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009d00 tx=0x7f80e8004d10 comp rx=0 tx=0).stop 2024-09-11T08:48:37.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 -- 172.21.15.43:0/3333264208 shutdown_connections 2024-09-11T08:48:37.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3333264208 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80f00fb660 0x7f80f0109520 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3333264208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f00facb0 0x7f80f00fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.321 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3333264208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80f0102790 0x7f80f0102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 -- 172.21.15.43:0/3333264208 >> 172.21.15.43:0/3333264208 conn(0x7f80f0074ba0 msgr2=0x7f80f00f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 -- 172.21.15.43:0/3333264208 shutdown_connections 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.319+0000 7f80f6a46640 1 -- 172.21.15.43:0/3333264208 wait complete. 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80f6a46640 1 Processor -- start 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80f6a46640 1 -- start start 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80f6a46640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f00facb0 0x7f80f0112a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80f6a46640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80f00fb660 0x7f80f0112f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80f6a46640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80f0102790 0x7f80f010cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80f6a46640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f010d230 con 0x7f80f00facb0 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80f6a46640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f010d3a0 con 0x7f80f0102790 2024-09-11T08:48:37.322 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80f6a46640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f010d560 con 0x7f80f00fb660 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80effff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f00facb0 0x7f80f0112a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80f00fb660 0x7f80f0112f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80effff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f00facb0 0x7f80f0112a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46462/0 (socket says 172.21.15.43:46462) 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.320+0000 7f80effff640 1 -- 172.21.15.43:0/3426283825 learned_addr learned my addr 172.21.15.43:0/3426283825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80f4fe3640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80f0102790 0x7f80f010cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80effff640 1 -- 172.21.15.43:0/3426283825 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80f00fb660 msgr2=0x7f80f0112f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80effff640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80f00fb660 0x7f80f0112f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80effff640 1 -- 172.21.15.43:0/3426283825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80f0102790 msgr2=0x7f80f010cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80effff640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80f0102790 0x7f80f010cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80effff640 1 -- 172.21.15.43:0/3426283825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e4005040 con 0x7f80f00facb0 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80f4fe3640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80f0102790 0x7f80f010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80effff640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f00facb0 0x7f80f0112a50 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b810 tx=0x7f80e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:37.323 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80ecff9640 1 -- 172.21.15.43:0/3426283825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e000be60 con 0x7f80f00facb0 2024-09-11T08:48:37.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e80098b0 con 0x7f80f00facb0 2024-09-11T08:48:37.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80ecff9640 1 -- 172.21.15.43:0/3426283825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f80e000cba0 con 0x7f80f00facb0 2024-09-11T08:48:37.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80ecff9640 1 -- 172.21.15.43:0/3426283825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0015b60 con 0x7f80f00facb0 2024-09-11T08:48:37.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.321+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f01198a0 con 0x7f80f00facb0 2024-09-11T08:48:37.325 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.322+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f00fdfd0 con 0x7f80f00facb0 2024-09-11T08:48:37.325 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.323+0000 7f80ecff9640 1 -- 172.21.15.43:0/3426283825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f80e0012070 con 0x7f80f00facb0 2024-09-11T08:48:37.325 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.323+0000 7f80ecff9640 1 --2- 172.21.15.43:0/3426283825 >> v2:172.21.15.43:6800/588375333 conn(0x7f80d806ef80 0x7f80d8071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:48:37.325 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.323+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/3426283825 >> v2:172.21.15.43:6800/588375333 conn(0x7f80d806ef80 0x7f80d8071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:48:37.325 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.323+0000 7f80ecff9640 1 -- 172.21.15.43:0/3426283825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(767..767 src has 1..767) v4 ==== 8046+0+0 (secure 0 0 0) 0x7f80e008e380 con 0x7f80f00facb0 2024-09-11T08:48:37.325 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.324+0000 7f80ef7fe640 1 --2- 172.21.15.43:0/3426283825 >> v2:172.21.15.43:6800/588375333 conn(0x7f80d806ef80 0x7f80d8071430 secure :-1 s=READY pgs=3483 cs=0 l=1 rev1=1 crypto rx=0x7f80f010e310 tx=0x7f80e400b040 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:48:37.327 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.328+0000 7f80ecff9640 1 -- 172.21.15.43:0/3426283825 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f80e005aa90 con 0x7f80f00facb0 2024-09-11T08:48:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]': finished 2024-09-11T08:48:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:37 smithi172 ceph-mon[32987]: osdmap e766: 8 total, 8 up, 8 in 2024-09-11T08:48:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:37 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1826338773' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:37 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:37 smithi172 ceph-mon[32987]: pgmap v1245: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-09-11T08:48:37.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:37.491 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:37.492+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f80f010bc30 con 0x7f80f00facb0 2024-09-11T08:48:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]': finished 2024-09-11T08:48:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[40592]: osdmap e766: 8 total, 8 up, 8 in 2024-09-11T08:48:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1826338773' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:37.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[40592]: pgmap v1245: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-09-11T08:48:37.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]': finished 2024-09-11T08:48:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[29355]: osdmap e766: 8 total, 8 up, 8 in 2024-09-11T08:48:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1826338773' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]: dispatch 2024-09-11T08:48:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[29355]: pgmap v1245: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-09-11T08:48:37.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:38.220 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:38.220+0000 7f80ecff9640 1 -- 172.21.15.43:0/3426283825 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 741af959-956f-4579-9aef-0637359b7771 v768) v1 ==== 223+0+0 (secure 0 0 0) 0x7f80e005a430 con 0x7f80f00facb0 2024-09-11T08:48:38.324 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:38.324+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f80f004bf60 con 0x7f80f00facb0 2024-09-11T08:48:38.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:38 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]': finished 2024-09-11T08:48:38.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:38 smithi172 ceph-mon[32987]: osdmap e767: 8 total, 8 up, 8 in 2024-09-11T08:48:38.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:38 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:48:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:38 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]': finished 2024-09-11T08:48:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:38 smithi043 ceph-mon[40592]: osdmap e767: 8 total, 8 up, 8 in 2024-09-11T08:48:38.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:38 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:48:38.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:38 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "741af959-956f-4579-9aef-0637359b7771", "app": "rados"}]': finished 2024-09-11T08:48:38.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:38 smithi043 ceph-mon[29355]: osdmap e767: 8 total, 8 up, 8 in 2024-09-11T08:48:38.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:38 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:48:39.234 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.234+0000 7f80ecff9640 1 -- 172.21.15.43:0/3426283825 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 741af959-956f-4579-9aef-0637359b7771 v769) v1 ==== 223+0+0 (secure 0 0 0) 0x7f80e005a610 con 0x7f80f00facb0 2024-09-11T08:48:39.234 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 10 for pool 741af959-956f-4579-9aef-0637359b7771 2024-09-11T08:48:39.237 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 >> v2:172.21.15.43:6800/588375333 conn(0x7f80d806ef80 msgr2=0x7f80d8071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:39.237 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3426283825 >> v2:172.21.15.43:6800/588375333 conn(0x7f80d806ef80 0x7f80d8071430 secure :-1 s=READY pgs=3483 cs=0 l=1 rev1=1 crypto rx=0x7f80f010e310 tx=0x7f80e400b040 comp rx=0 tx=0).stop 2024-09-11T08:48:39.237 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f00facb0 msgr2=0x7f80f0112a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:48:39.237 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f00facb0 0x7f80f0112a50 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b810 tx=0x7f80e00091d0 comp rx=0 tx=0).stop 2024-09-11T08:48:39.237 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 shutdown_connections 2024-09-11T08:48:39.237 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3426283825 >> v2:172.21.15.43:6800/588375333 conn(0x7f80d806ef80 0x7f80d8071430 unknown :-1 s=CLOSED pgs=3483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:39.237 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80f00fb660 0x7f80f0112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:39.237 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80f0102790 0x7f80f010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:39.238 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 --2- 172.21.15.43:0/3426283825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80f00facb0 0x7f80f0112a50 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:48:39.238 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 >> 172.21.15.43:0/3426283825 conn(0x7f80f0074ba0 msgr2=0x7f80f00f9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:48:39.238 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 shutdown_connections 2024-09-11T08:48:39.238 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:48:39.237+0000 7f80f6a46640 1 -- 172.21.15.43:0/3426283825 wait complete. 2024-09-11T08:48:39.248 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-11T08:48:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:48:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:39 smithi043 ceph-mon[40592]: osdmap e768: 8 total, 8 up, 8 in 2024-09-11T08:48:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:39 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:48:39.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:39 smithi043 ceph-mon[40592]: pgmap v1248: 188 pgs: 3 unknown, 185 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:39.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:48:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:39 smithi043 ceph-mon[29355]: osdmap e768: 8 total, 8 up, 8 in 2024-09-11T08:48:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:39 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:48:39.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:39 smithi043 ceph-mon[29355]: pgmap v1248: 188 pgs: 3 unknown, 185 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:48:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:39 smithi172 ceph-mon[32987]: osdmap e768: 8 total, 8 up, 8 in 2024-09-11T08:48:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:39 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:48:39.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:39 smithi172 ceph-mon[32987]: pgmap v1248: 188 pgs: 3 unknown, 185 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:40.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:48:40.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:40 smithi043 ceph-mon[40592]: osdmap e769: 8 total, 8 up, 8 in 2024-09-11T08:48:40.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:48:40.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:40 smithi043 ceph-mon[29355]: osdmap e769: 8 total, 8 up, 8 in 2024-09-11T08:48:40.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3426283825' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "741af959-956f-4579-9aef-0637359b7771", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:48:40.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:40 smithi172 ceph-mon[32987]: osdmap e769: 8 total, 8 up, 8 in 2024-09-11T08:48:41.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:41 smithi043 ceph-mon[40592]: pgmap v1250: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:41.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:41 smithi043 ceph-mon[29355]: pgmap v1250: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:41 smithi172 ceph-mon[32987]: pgmap v1250: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:48:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:43 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:43 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:43.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:43 smithi043 ceph-mon[40592]: pgmap v1251: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:48:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:43 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:43 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:43.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:43 smithi043 ceph-mon[29355]: pgmap v1251: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:48:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:43 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:43 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:43.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:43 smithi172 ceph-mon[32987]: pgmap v1251: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:48:44.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:48:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:43] "GET /metrics HTTP/1.1" 200 228715 "" "Prometheus/2.43.0" 2024-09-11T08:48:45.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:45 smithi043 ceph-mon[40592]: pgmap v1252: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2024-09-11T08:48:45.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:45 smithi043 ceph-mon[29355]: pgmap v1252: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2024-09-11T08:48:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:45 smithi172 ceph-mon[32987]: pgmap v1252: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2024-09-11T08:48:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:48:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:46.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:48:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:48:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:47 smithi043 ceph-mon[40592]: pgmap v1253: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:48:47.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:47 smithi043 ceph-mon[29355]: pgmap v1253: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:48:47.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:47 smithi172 ceph-mon[32987]: pgmap v1253: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:49.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:49 smithi172 ceph-mon[32987]: pgmap v1254: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:49.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:49 smithi043 ceph-mon[40592]: pgmap v1254: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:49.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:49 smithi043 ceph-mon[29355]: pgmap v1254: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:48:51.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:51 smithi043 ceph-mon[40592]: pgmap v1255: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-11T08:48:51.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:51 smithi043 ceph-mon[29355]: pgmap v1255: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-11T08:48:51.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:51 smithi172 ceph-mon[32987]: pgmap v1255: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-11T08:48:53.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:53 smithi043 ceph-mon[40592]: pgmap v1256: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:53.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:53 smithi043 ceph-mon[29355]: pgmap v1256: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:53.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:53 smithi172 ceph-mon[32987]: pgmap v1256: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:54.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:48:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:53] "GET /metrics HTTP/1.1" 200 228715 "" "Prometheus/2.43.0" 2024-09-11T08:48:55.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:55 smithi043 ceph-mon[40592]: pgmap v1257: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:55.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:55 smithi043 ceph-mon[29355]: pgmap v1257: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:55 smithi172 ceph-mon[32987]: pgmap v1257: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:48:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:48:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:56.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:48:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:48:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:57 smithi043 ceph-mon[40592]: pgmap v1258: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:57.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:57.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:57 smithi043 ceph-mon[29355]: pgmap v1258: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:57.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:57 smithi172 ceph-mon[32987]: pgmap v1258: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:48:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:59.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:48:59 smithi172 ceph-mon[32987]: pgmap v1259: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:48:59 smithi043 ceph-mon[40592]: pgmap v1259: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:48:59.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:48:59 smithi043 ceph-mon[29355]: pgmap v1259: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:01.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:01 smithi172 ceph-mon[32987]: pgmap v1260: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:01 smithi043 ceph-mon[40592]: pgmap v1260: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:01 smithi043 ceph-mon[29355]: pgmap v1260: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:03.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:03 smithi172 ceph-mon[32987]: pgmap v1261: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:03.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:03 smithi043 ceph-mon[40592]: pgmap v1261: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:03 smithi043 ceph-mon[29355]: pgmap v1261: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:04.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:49:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:03] "GET /metrics HTTP/1.1" 200 228703 "" "Prometheus/2.43.0" 2024-09-11T08:49:05.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:05 smithi172 ceph-mon[32987]: pgmap v1262: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:05.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:49:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:05.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:05 smithi043 ceph-mon[40592]: pgmap v1262: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:05.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:05 smithi043 ceph-mon[29355]: pgmap v1262: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:06.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:49:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:49:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:07 smithi172 ceph-mon[32987]: pgmap v1263: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:07.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:07.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:07 smithi043 ceph-mon[29355]: pgmap v1263: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:07.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:07 smithi043 ceph-mon[40592]: pgmap v1263: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:09.250 INFO:tasks.workunit.client.0.smithi043.stderr:++ seq 1 10 2024-09-11T08:49:09.251 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.251 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj1 /etc/passwd 2024-09-11T08:49:09.289 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.290 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj2 /etc/passwd 2024-09-11T08:49:09.324 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.325 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj3 /etc/passwd 2024-09-11T08:49:09.358 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.359 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj4 /etc/passwd 2024-09-11T08:49:09.391 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.391 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj5 /etc/passwd 2024-09-11T08:49:09.425 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.426 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj6 /etc/passwd 2024-09-11T08:49:09.457 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.457 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj7 /etc/passwd 2024-09-11T08:49:09.489 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.490 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj8 /etc/passwd 2024-09-11T08:49:09.521 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.521 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj9 /etc/passwd 2024-09-11T08:49:09.553 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-11T08:49:09.553 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 741af959-956f-4579-9aef-0637359b7771 put obj10 /etc/passwd 2024-09-11T08:49:09.585 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-11T08:49:09.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:09 smithi172 ceph-mon[32987]: pgmap v1264: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:09 smithi043 ceph-mon[40592]: pgmap v1264: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:09 smithi043 ceph-mon[29355]: pgmap v1264: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:11.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:11 smithi172 ceph-mon[32987]: pgmap v1265: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:11.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:11 smithi043 ceph-mon[40592]: pgmap v1265: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:11 smithi043 ceph-mon[29355]: pgmap v1265: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:12.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:13.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:13 smithi172 ceph-mon[32987]: pgmap v1266: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:13 smithi043 ceph-mon[40592]: pgmap v1266: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:13 smithi043 ceph-mon[29355]: pgmap v1266: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:14.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:49:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:13] "GET /metrics HTTP/1.1" 200 228663 "" "Prometheus/2.43.0" 2024-09-11T08:49:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:15 smithi172 ceph-mon[32987]: pgmap v1267: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:49:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:49:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:15 smithi043 ceph-mon[40592]: pgmap v1267: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:49:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:15 smithi043 ceph-mon[29355]: pgmap v1267: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:49:16.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:49:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:49:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:17 smithi172 ceph-mon[32987]: pgmap v1268: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-11T08:49:17.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:17 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:17 smithi043 ceph-mon[40592]: pgmap v1268: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-11T08:49:17.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:17 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:17 smithi043 ceph-mon[29355]: pgmap v1268: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-11T08:49:17.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:17 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:18 smithi172 ceph-mon[32987]: pool '741af959-956f-4579-9aef-0637359b7771' is full (reached quota's max_objects: 10) 2024-09-11T08:49:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:18 smithi172 ceph-mon[32987]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:49:18.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:18 smithi172 ceph-mon[32987]: osdmap e770: 8 total, 8 up, 8 in 2024-09-11T08:49:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:18 smithi043 ceph-mon[40592]: pool '741af959-956f-4579-9aef-0637359b7771' is full (reached quota's max_objects: 10) 2024-09-11T08:49:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:18 smithi043 ceph-mon[40592]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:49:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:18 smithi043 ceph-mon[40592]: osdmap e770: 8 total, 8 up, 8 in 2024-09-11T08:49:18.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:18 smithi043 ceph-mon[29355]: pool '741af959-956f-4579-9aef-0637359b7771' is full (reached quota's max_objects: 10) 2024-09-11T08:49:18.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:18 smithi043 ceph-mon[29355]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:49:18.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:18 smithi043 ceph-mon[29355]: osdmap e770: 8 total, 8 up, 8 in 2024-09-11T08:49:19.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:19 smithi172 ceph-mon[32987]: pgmap v1270: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:19.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:19 smithi043 ceph-mon[40592]: pgmap v1270: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:19.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:19 smithi043 ceph-mon[29355]: pgmap v1270: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:21.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:21 smithi172 ceph-mon[32987]: pgmap v1271: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:21.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:21 smithi043 ceph-mon[40592]: pgmap v1271: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:21.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:21 smithi043 ceph-mon[29355]: pgmap v1271: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:23.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:23 smithi172 ceph-mon[32987]: pgmap v1272: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:23 smithi043 ceph-mon[40592]: pgmap v1272: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:23 smithi043 ceph-mon[29355]: pgmap v1272: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:49:24.144 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:49:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:23] "GET /metrics HTTP/1.1" 200 228663 "" "Prometheus/2.43.0" 2024-09-11T08:49:25.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:25 smithi172 ceph-mon[32987]: pgmap v1273: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:49:25.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:49:25 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:25.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:25 smithi043 ceph-mon[29355]: pgmap v1273: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:49:25.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:25 smithi043 ceph-mon[40592]: pgmap v1273: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:49:26.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:49:26 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:49:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:27 smithi172 ceph-mon[32987]: pgmap v1274: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:49:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:27 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:27 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:27.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:27 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:27 smithi043 ceph-mon[40592]: pgmap v1274: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:49:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:27 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:27 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:27.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:27 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:27 smithi043 ceph-mon[29355]: pgmap v1274: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:49:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:27 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:27.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:27 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:27.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:27 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:29.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:29 smithi172 ceph-mon[32987]: pgmap v1275: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:49:29.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:29 smithi043 ceph-mon[40592]: pgmap v1275: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:49:29.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:29 smithi043 ceph-mon[29355]: pgmap v1275: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T08:49:31.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:31 smithi172 ceph-mon[32987]: pgmap v1276: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:31.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:31 smithi043 ceph-mon[40592]: pgmap v1276: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:31.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:31 smithi043 ceph-mon[29355]: pgmap v1276: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:32 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:49:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:32 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:49:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:32 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:49:32.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:32 smithi172 ceph-mon[32987]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:32.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:32 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:49:32.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:32 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:49:32.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:32 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:49:32.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:32 smithi043 ceph-mon[29355]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:32 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:49:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:32 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:49:32.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:32 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:49:32.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:32 smithi043 ceph-mon[40592]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:33.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:33 smithi172 ceph-mon[32987]: pgmap v1277: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:33 smithi043 ceph-mon[29355]: pgmap v1277: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:33 smithi043 ceph-mon[40592]: pgmap v1277: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:34.143 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:49:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:33] "GET /metrics HTTP/1.1" 200 228760 "" "Prometheus/2.43.0" 2024-09-11T08:49:35.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:35 smithi172 ceph-mon[32987]: pgmap v1278: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:35.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:49:35 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:35.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:35 smithi043 ceph-mon[29355]: pgmap v1278: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:35.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:35 smithi043 ceph-mon[40592]: pgmap v1278: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:36.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:49:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:49:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:37 smithi172 ceph-mon[32987]: pgmap v1279: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:37.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:37 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:37 smithi043 ceph-mon[40592]: pgmap v1279: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:37.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:37 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:37 smithi043 ceph-mon[29355]: pgmap v1279: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:37.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:37 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:39.586 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 992c53e8-268f-42a0-a098-268d5016ad85 put threemore /etc/passwd 2024-09-11T08:49:39.624 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 992c53e8-268f-42a0-a098-268d5016ad85 max_bytes 0 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.709+0000 7f5c66909640 1 -- 172.21.15.43:0/1525829886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c60103070 msgr2=0x7f5c600fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.709+0000 7f5c66909640 1 --2- 172.21.15.43:0/1525829886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c60103070 0x7f5c600fc740 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f5c54004980 tx=0x7f5c54009da0 comp rx=0 tx=0).stop 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 -- 172.21.15.43:0/1525829886 shutdown_connections 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 --2- 172.21.15.43:0/1525829886 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5c600fce70 0x7f5c60109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 --2- 172.21.15.43:0/1525829886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c60103070 0x7f5c600fc740 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 --2- 172.21.15.43:0/1525829886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c601026d0 0x7f5c60102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 -- 172.21.15.43:0/1525829886 >> 172.21.15.43:0/1525829886 conn(0x7f5c600f82e0 msgr2=0x7f5c600fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 -- 172.21.15.43:0/1525829886 shutdown_connections 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 -- 172.21.15.43:0/1525829886 wait complete. 2024-09-11T08:49:39.711 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 Processor -- start 2024-09-11T08:49:39.712 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.710+0000 7f5c66909640 1 -- start start 2024-09-11T08:49:39.714 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c66909640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c600fce70 0x7f5c6006f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:49:39.714 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c66909640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5c601026d0 0x7f5c6006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:49:39.714 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c66909640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60103070 0x7f5c600703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c66909640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6011b2a0 con 0x7f5c600fce70 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c66909640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6011b3e0 con 0x7f5c60103070 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c66909640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6011b550 con 0x7f5c601026d0 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c600fce70 0x7f5c6006f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5f7fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5c601026d0 0x7f5c6006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c600fce70 0x7f5c6006f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57836/0 (socket says 172.21.15.43:57836) 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5f7fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5c601026d0 0x7f5c6006fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49160/0 (socket says 172.21.15.43:49160) 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 -- 172.21.15.43:0/1724368451 learned_addr learned my addr 172.21.15.43:0/1724368451 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c64ea6640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60103070 0x7f5c600703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 -- 172.21.15.43:0/1724368451 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5c601026d0 msgr2=0x7f5c6006fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5c601026d0 0x7f5c6006fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:39.715 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 -- 172.21.15.43:0/1724368451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60103070 msgr2=0x7f5c600703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60103070 0x7f5c600703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580098b0 con 0x7f5c600fce70 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c64ea6640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60103070 0x7f5c600703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5f7fe640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5c601026d0 0x7f5c6006fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5ffff640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c600fce70 0x7f5c6006f940 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000c8b0 tx=0x7f5c50004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000cea0 con 0x7f5c600fce70 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f5c5000acc0 con 0x7f5c600fce70 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.711+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50012bf0 con 0x7f5c600fce70 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.712+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c54005040 con 0x7f5c600fce70 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.712+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6011bbc0 con 0x7f5c600fce70 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.712+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c600fde80 con 0x7f5c600fce70 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.713+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f5c5000a850 con 0x7f5c600fce70 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.714+0000 7f5c5cff9640 1 --2- 172.21.15.43:0/1724368451 >> v2:172.21.15.43:6800/588375333 conn(0x7f5c4806ef80 0x7f5c48071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:49:39.716 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.714+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(770..770 src has 1..770) v4 ==== 8046+0+0 (secure 0 0 0) 0x7f5c5008eb00 con 0x7f5c600fce70 2024-09-11T08:49:39.717 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.714+0000 7f5c5f7fe640 1 --2- 172.21.15.43:0/1724368451 >> v2:172.21.15.43:6800/588375333 conn(0x7f5c4806ef80 0x7f5c48071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:49:39.717 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.714+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=771}) v3 -- 0x7f5c4807aa20 con 0x7f5c600fce70 2024-09-11T08:49:39.717 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.714+0000 7f5c5f7fe640 1 --2- 172.21.15.43:0/1724368451 >> v2:172.21.15.43:6800/588375333 conn(0x7f5c4806ef80 0x7f5c48071430 secure :-1 s=READY pgs=3495 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009040 tx=0x7f5c54009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:49:39.718 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.718+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5c5005b370 con 0x7f5c600fce70 2024-09-11T08:49:39.730 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:39 smithi172 ceph-mon[32987]: pgmap v1280: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:39.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:39.886+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f5c6010bb60 con 0x7f5c600fce70 2024-09-11T08:49:39.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:39 smithi043 ceph-mon[40592]: pgmap v1280: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:39.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:39 smithi043 ceph-mon[29355]: pgmap v1280: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:40.465 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:40.465+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 7 ==== osd_map(771..771 src has 1..771) v4 ==== 620+0+0 (secure 0 0 0) 0x7f5c5000ab00 con 0x7f5c600fce70 2024-09-11T08:49:40.466 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:40.465+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=772}) v3 -- 0x7f5c4807b5d0 con 0x7f5c600fce70 2024-09-11T08:49:40.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:40.470+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v771) v1 ==== 217+0+0 (secure 0 0 0) 0x7f5c5005ad10 con 0x7f5c600fce70 2024-09-11T08:49:40.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:40.576+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f5c6010bb60 con 0x7f5c600fce70 2024-09-11T08:49:40.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:40 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:49:40.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:40 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:49:40.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:40 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:49:41.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.466+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 9 ==== osd_map(772..772 src has 1..772) v4 ==== 620+0+0 (secure 0 0 0) 0x7f5c50012400 con 0x7f5c600fce70 2024-09-11T08:49:41.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.466+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=773}) v3 -- 0x7f5c4807bae0 con 0x7f5c600fce70 2024-09-11T08:49:41.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.470+0000 7f5c5cff9640 1 -- 172.21.15.43:0/1724368451 <== mon.0 v2:172.21.15.43:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v772) v1 ==== 217+0+0 (secure 0 0 0) 0x7f5c5005aef0 con 0x7f5c600fce70 2024-09-11T08:49:41.471 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 >> v2:172.21.15.43:6800/588375333 conn(0x7f5c4806ef80 msgr2=0x7f5c48071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 --2- 172.21.15.43:0/1724368451 >> v2:172.21.15.43:6800/588375333 conn(0x7f5c4806ef80 0x7f5c48071430 secure :-1 s=READY pgs=3495 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009040 tx=0x7f5c54009da0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c600fce70 msgr2=0x7f5c6006f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c600fce70 0x7f5c6006f940 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000c8b0 tx=0x7f5c50004430 comp rx=0 tx=0).stop 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 shutdown_connections 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 --2- 172.21.15.43:0/1724368451 >> v2:172.21.15.43:6800/588375333 conn(0x7f5c4806ef80 0x7f5c48071430 unknown :-1 s=CLOSED pgs=3495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5c601026d0 0x7f5c6006fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60103070 0x7f5c600703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 --2- 172.21.15.43:0/1724368451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c600fce70 0x7f5c6006f940 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 >> 172.21.15.43:0/1724368451 conn(0x7f5c600f82e0 msgr2=0x7f5c60100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:49:41.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 shutdown_connections 2024-09-11T08:49:41.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.474+0000 7f5c66909640 1 -- 172.21.15.43:0/1724368451 wait complete. 2024-09-11T08:49:41.487 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 992c53e8-268f-42a0-a098-268d5016ad85 max_objects 0 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 -- 172.21.15.43:0/407229000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc102730 msgr2=0x7fcabc102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 --2- 172.21.15.43:0/407229000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc102730 0x7fcabc102b00 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fcab8004890 tx=0x7fcab8009da0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 -- 172.21.15.43:0/407229000 shutdown_connections 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 --2- 172.21.15.43:0/407229000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcabc0fb360 0x7fcabc109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 --2- 172.21.15.43:0/407229000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcabc1030d0 0x7fcabc0fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 --2- 172.21.15.43:0/407229000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc102730 0x7fcabc102b00 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 -- 172.21.15.43:0/407229000 >> 172.21.15.43:0/407229000 conn(0x7fcabc074ba0 msgr2=0x7fcabc0f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 -- 172.21.15.43:0/407229000 shutdown_connections 2024-09-11T08:49:41.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.572+0000 7fcac40f8640 1 -- 172.21.15.43:0/407229000 wait complete. 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac40f8640 1 Processor -- start 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac40f8640 1 -- start start 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac40f8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcabc0fb360 0x7fcabc112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac40f8640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcabc102730 0x7fcabc113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac40f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc1030d0 0x7fcabc10cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac40f8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc10d390 con 0x7fcabc1030d0 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac40f8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc10d500 con 0x7fcabc0fb360 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac40f8640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc10d6c0 con 0x7fcabc102730 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac1693640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcabc102730 0x7fcabc113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac2695640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc1030d0 0x7fcabc10cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac1693640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcabc102730 0x7fcabc113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:49176/0 (socket says 172.21.15.43:49176) 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac1693640 1 -- 172.21.15.43:0/4191272374 learned_addr learned my addr 172.21.15.43:0/4191272374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:49:41.576 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac2695640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc1030d0 0x7fcabc10cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57848/0 (socket says 172.21.15.43:57848) 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.573+0000 7fcac1e94640 1 --2- 172.21.15.43:0/4191272374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcabc0fb360 0x7fcabc112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcac1693640 1 -- 172.21.15.43:0/4191272374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcabc0fb360 msgr2=0x7fcabc112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcac1693640 1 --2- 172.21.15.43:0/4191272374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcabc0fb360 0x7fcabc112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcac1693640 1 -- 172.21.15.43:0/4191272374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc1030d0 msgr2=0x7fcabc10cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcac1693640 1 --2- 172.21.15.43:0/4191272374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc1030d0 0x7fcabc10cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcac1693640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40098b0 con 0x7fcabc102730 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcac1693640 1 --2- 172.21.15.43:0/4191272374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcabc102730 0x7fcabc113100 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0028c0 tx=0x7fcaac004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac007c30 con 0x7fcabc102730 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fcaac00f040 con 0x7fcabc102730 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab8005040 con 0x7fcabc102730 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac009340 con 0x7fcabc102730 2024-09-11T08:49:41.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.574+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc1199e0 con 0x7fcabc102730 2024-09-11T08:49:41.578 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.575+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcabc0fdf50 con 0x7fcabc102730 2024-09-11T08:49:41.578 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.576+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fcaac009770 con 0x7fcabc102730 2024-09-11T08:49:41.579 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.576+0000 7fcab27fc640 1 --2- 172.21.15.43:0/4191272374 >> v2:172.21.15.43:6800/588375333 conn(0x7fcaa806ef80 0x7fcaa8071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:49:41.579 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.576+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(772..772 src has 1..772) v4 ==== 8046+0+0 (secure 0 0 0) 0x7fcaac08eaf0 con 0x7fcabc102730 2024-09-11T08:49:41.579 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.576+0000 7fcac1e94640 1 --2- 172.21.15.43:0/4191272374 >> v2:172.21.15.43:6800/588375333 conn(0x7fcaa806ef80 0x7fcaa8071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:49:41.579 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.576+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=773}) v3 -- 0x7fcaa807a9a0 con 0x7fcabc102730 2024-09-11T08:49:41.579 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.577+0000 7fcac1e94640 1 --2- 172.21.15.43:0/4191272374 >> v2:172.21.15.43:6800/588375333 conn(0x7fcaa806ef80 0x7fcaa8071430 secure :-1 s=READY pgs=3496 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009040 tx=0x7fcab8009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:49:41.581 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.580+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcaac05b2b0 con 0x7fcabc102730 2024-09-11T08:49:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:41 smithi172 ceph-mon[32987]: pgmap v1281: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:49:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:41 smithi172 ceph-mon[32987]: osdmap e771: 8 total, 8 up, 8 in 2024-09-11T08:49:41.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:41 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:49:41.745 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:41.745+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fcabc10e200 con 0x7fcabc102730 2024-09-11T08:49:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:41 smithi043 ceph-mon[40592]: pgmap v1281: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:49:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:41 smithi043 ceph-mon[40592]: osdmap e771: 8 total, 8 up, 8 in 2024-09-11T08:49:41.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:41 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:49:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:41 smithi043 ceph-mon[29355]: pgmap v1281: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:49:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:49:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:41 smithi043 ceph-mon[29355]: osdmap e771: 8 total, 8 up, 8 in 2024-09-11T08:49:41.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:41 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:49:42.484 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:42.483+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(773..773 src has 1..773) v4 ==== 620+0+0 (secure 0 0 0) 0x7fcaac007d90 con 0x7fcabc102730 2024-09-11T08:49:42.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:42.484+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=774}) v3 -- 0x7fcaa807b5b0 con 0x7fcabc102730 2024-09-11T08:49:42.490 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:42.489+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v773) v1 ==== 221+0+0 (secure 0 0 0) 0x7fcaac05ac50 con 0x7fcabc102730 2024-09-11T08:49:42.592 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:42.591+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fcabc10e200 con 0x7fcabc102730 2024-09-11T08:49:42.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:49:42.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:42 smithi172 ceph-mon[32987]: osdmap e772: 8 total, 8 up, 8 in 2024-09-11T08:49:42.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:42 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4191272374' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:42.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:42 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:42.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:42 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:49:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[40592]: osdmap e772: 8 total, 8 up, 8 in 2024-09-11T08:49:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4191272374' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:42.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:42.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/1724368451' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:49:42.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[29355]: osdmap e772: 8 total, 8 up, 8 in 2024-09-11T08:49:42.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4191272374' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:42.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:42 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:43.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.487+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 9 ==== osd_map(774..774 src has 1..774) v4 ==== 620+0+0 (secure 0 0 0) 0x7fcaac05aff0 con 0x7fcabc102730 2024-09-11T08:49:43.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.487+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=775}) v3 -- 0x7fcaa807bac0 con 0x7fcabc102730 2024-09-11T08:49:43.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.492+0000 7fcab27fc640 1 -- 172.21.15.43:0/4191272374 <== mon.2 v2:172.21.15.43:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 v774) v1 ==== 221+0+0 (secure 0 0 0) 0x7fcaac05ac50 con 0x7fcabc102730 2024-09-11T08:49:43.492 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 0 for pool 992c53e8-268f-42a0-a098-268d5016ad85 2024-09-11T08:49:43.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.495+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 >> v2:172.21.15.43:6800/588375333 conn(0x7fcaa806ef80 msgr2=0x7fcaa8071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.495+0000 7fcac40f8640 1 --2- 172.21.15.43:0/4191272374 >> v2:172.21.15.43:6800/588375333 conn(0x7fcaa806ef80 0x7fcaa8071430 secure :-1 s=READY pgs=3496 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009040 tx=0x7fcab8009da0 comp rx=0 tx=0).stop 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcabc102730 msgr2=0x7fcabc113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 --2- 172.21.15.43:0/4191272374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcabc102730 0x7fcabc113100 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0028c0 tx=0x7fcaac004570 comp rx=0 tx=0).stop 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 shutdown_connections 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 --2- 172.21.15.43:0/4191272374 >> v2:172.21.15.43:6800/588375333 conn(0x7fcaa806ef80 0x7fcaa8071430 unknown :-1 s=CLOSED pgs=3496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 --2- 172.21.15.43:0/4191272374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcabc102730 0x7fcabc113100 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 --2- 172.21.15.43:0/4191272374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcabc1030d0 0x7fcabc10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 --2- 172.21.15.43:0/4191272374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcabc0fb360 0x7fcabc112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 >> 172.21.15.43:0/4191272374 conn(0x7fcabc074ba0 msgr2=0x7fcabc1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 shutdown_connections 2024-09-11T08:49:43.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:49:43.496+0000 7fcac40f8640 1 -- 172.21.15.43:0/4191272374 wait complete. 2024-09-11T08:49:43.507 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-11T08:49:43.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:43] "GET /metrics HTTP/1.1" 200 228724 "" "Prometheus/2.43.0" 2024-09-11T08:49:43.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[40592]: pgmap v1284: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[40592]: osdmap e773: 8 total, 8 up, 8 in 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/4191272374' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[29355]: pgmap v1284: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[29355]: osdmap e773: 8 total, 8 up, 8 in 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/4191272374' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:43.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:43 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:43 smithi172 ceph-mon[32987]: pgmap v1284: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:49:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:49:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:43 smithi172 ceph-mon[32987]: osdmap e773: 8 total, 8 up, 8 in 2024-09-11T08:49:43.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:43 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/4191272374' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:43.982 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:43 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:49:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:44 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:49:44.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:44 smithi043 ceph-mon[40592]: osdmap e774: 8 total, 8 up, 8 in 2024-09-11T08:49:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:44 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:49:44.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:44 smithi043 ceph-mon[29355]: osdmap e774: 8 total, 8 up, 8 in 2024-09-11T08:49:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:44 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:49:44.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:44 smithi172 ceph-mon[32987]: osdmap e774: 8 total, 8 up, 8 in 2024-09-11T08:49:45.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:45 smithi172 ceph-mon[32987]: pgmap v1287: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:49:45.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:49:45 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:45.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:45 smithi043 ceph-mon[40592]: pgmap v1287: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:49:45.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:45 smithi043 ceph-mon[29355]: pgmap v1287: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:49:46.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:49:46 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:49:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:47 smithi043 ceph-mon[40592]: pgmap v1288: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 870 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:49:47.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:47 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:47.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:47 smithi043 ceph-mon[29355]: pgmap v1288: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 870 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:49:47.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:47 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:47 smithi172 ceph-mon[32987]: pgmap v1288: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 870 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:49:47.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:47 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:49.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:49 smithi043 ceph-mon[29355]: pgmap v1289: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-11T08:49:49.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:49 smithi043 ceph-mon[40592]: pgmap v1289: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-11T08:49:49.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:49 smithi172 ceph-mon[32987]: pgmap v1289: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-11T08:49:51.783 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:51 smithi172 ceph-mon[32987]: pgmap v1290: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:49:51.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:51 smithi043 ceph-mon[29355]: pgmap v1290: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:49:51.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:51 smithi043 ceph-mon[40592]: pgmap v1290: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:49:53.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:53 smithi043 ceph-mon[40592]: pgmap v1291: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-11T08:49:53.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:53 smithi043 ceph-mon[29355]: pgmap v1291: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-11T08:49:53.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:49:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:53] "GET /metrics HTTP/1.1" 200 228724 "" "Prometheus/2.43.0" 2024-09-11T08:49:53.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:53 smithi172 ceph-mon[32987]: pgmap v1291: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-11T08:49:55.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:55 smithi172 ceph-mon[32987]: pgmap v1292: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-09-11T08:49:55.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:49:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:49:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:55 smithi043 ceph-mon[40592]: pgmap v1292: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-09-11T08:49:55.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:55 smithi043 ceph-mon[29355]: pgmap v1292: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-09-11T08:49:56.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:49:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:49:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:57 smithi043 ceph-mon[40592]: pgmap v1293: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:57 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:57.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:57 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:57 smithi043 ceph-mon[29355]: pgmap v1293: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:57 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:57.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:57 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:57.980 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:57 smithi172 ceph-mon[32987]: pgmap v1293: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:57 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:57.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:57 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:59.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:49:59 smithi043 ceph-mon[40592]: pgmap v1294: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:59.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:49:59 smithi043 ceph-mon[29355]: pgmap v1294: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:59.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:49:59 smithi172 ceph-mon[32987]: pgmap v1294: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[40592]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-11T08:50:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[40592]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:50:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[40592]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:50:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[40592]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:50:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[40592]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-11T08:50:00.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[40592]: pool '741af959-956f-4579-9aef-0637359b7771' is full (running out of quota) 2024-09-11T08:50:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[29355]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-11T08:50:00.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[29355]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:50:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[29355]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:50:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[29355]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:50:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[29355]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-11T08:50:00.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:00 smithi043 ceph-mon[29355]: pool '741af959-956f-4579-9aef-0637359b7771' is full (running out of quota) 2024-09-11T08:50:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:00 smithi172 ceph-mon[32987]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-11T08:50:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:00 smithi172 ceph-mon[32987]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:50:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:00 smithi172 ceph-mon[32987]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:50:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:00 smithi172 ceph-mon[32987]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:50:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:00 smithi172 ceph-mon[32987]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-11T08:50:00.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:00 smithi172 ceph-mon[32987]: pool '741af959-956f-4579-9aef-0637359b7771' is full (running out of quota) 2024-09-11T08:50:01.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:01 smithi043 ceph-mon[40592]: pgmap v1295: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:01.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:01 smithi043 ceph-mon[29355]: pgmap v1295: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:01.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:01 smithi172 ceph-mon[32987]: pgmap v1295: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:03 smithi043 ceph-mon[40592]: pgmap v1296: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:50:03.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:50:03] "GET /metrics HTTP/1.1" 200 228741 "" "Prometheus/2.43.0" 2024-09-11T08:50:03.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:03 smithi043 ceph-mon[29355]: pgmap v1296: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:50:03.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:03 smithi172 ceph-mon[32987]: pgmap v1296: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:50:05.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:05 smithi172 ceph-mon[32987]: pgmap v1297: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:05.732 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:50:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:50:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:50:05.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:05 smithi043 ceph-mon[40592]: pgmap v1297: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:05.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:05 smithi043 ceph-mon[29355]: pgmap v1297: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:06.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:50:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:50:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:07 smithi043 ceph-mon[40592]: pgmap v1298: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:07.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:07 smithi043 ceph-mon[40592]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:07.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:07 smithi043 ceph-mon[29355]: pgmap v1298: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:07.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:07 smithi043 ceph-mon[29355]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:07 smithi172 ceph-mon[32987]: pgmap v1298: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:07.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:07 smithi172 ceph-mon[32987]: from='client.14514 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:09.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:09 smithi043 ceph-mon[29355]: pgmap v1299: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:09.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:09 smithi043 ceph-mon[40592]: pgmap v1299: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:09.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:09 smithi172 ceph-mon[32987]: pgmap v1299: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:11.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:11 smithi043 ceph-mon[40592]: pgmap v1300: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:11.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:11 smithi043 ceph-mon[29355]: pgmap v1300: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:11.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:11 smithi172 ceph-mon[32987]: pgmap v1300: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:50:12.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:12 smithi043 ceph-mon[40592]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:12.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:12 smithi043 ceph-mon[29355]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:12.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:12 smithi172 ceph-mon[32987]: from='mgr.14553 172.21.15.43:0/2741894141' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:13.508 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool delete 992c53e8-268f-42a0-a098-268d5016ad85 992c53e8-268f-42a0-a098-268d5016ad85 --yes-i-really-really-mean-it 2024-09-11T08:50:13.604 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 -- 172.21.15.43:0/2251881936 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 msgr2=0x7f37741083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:13.604 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 --2- 172.21.15.43:0/2251881936 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 0x7f37741083d0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f376c009d20 tx=0x7f376c004c50 comp rx=0 tx=0).stop 2024-09-11T08:50:13.604 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 -- 172.21.15.43:0/2251881936 shutdown_connections 2024-09-11T08:50:13.604 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 --2- 172.21.15.43:0/2251881936 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 0x7f37741083d0 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:13.604 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 --2- 172.21.15.43:0/2251881936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3774101f90 0x7f37740fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 --2- 172.21.15.43:0/2251881936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37741015f0 0x7f37741019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 -- 172.21.15.43:0/2251881936 >> 172.21.15.43:0/2251881936 conn(0x7f3774072530 msgr2=0x7f37740f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 -- 172.21.15.43:0/2251881936 shutdown_connections 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.602+0000 7f3779ea8640 1 -- 172.21.15.43:0/2251881936 wait complete. 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3779ea8640 1 Processor -- start 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3779ea8640 1 -- start start 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3779ea8640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 0x7f377410dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3779ea8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37741015f0 0x7f377410e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3779ea8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774101f90 0x7f377410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3779ea8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3774118e70 con 0x7f37741015f0 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3779ea8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3774118fe0 con 0x7f3774101f90 2024-09-11T08:50:13.605 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3779ea8640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37741191a0 con 0x7f37740fbd30 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f3772ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37741015f0 0x7f377410e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.603+0000 7f37737fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 0x7f377410dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3772ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37741015f0 0x7f377410e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40546/0 (socket says 172.21.15.43:40546) 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f37737fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 0x7f377410dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:60032/0 (socket says 172.21.15.43:60032) 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3772ffd640 1 -- 172.21.15.43:0/3236725877 learned_addr learned my addr 172.21.15.43:0/3236725877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3772ffd640 1 -- 172.21.15.43:0/3236725877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 msgr2=0x7f377410dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3772ffd640 1 --2- 172.21.15.43:0/3236725877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 0x7f377410dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3772ffd640 1 -- 172.21.15.43:0/3236725877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774101f90 msgr2=0x7f377410e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3772ffd640 1 --2- 172.21.15.43:0/3236725877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774101f90 0x7f377410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3772ffd640 1 -- 172.21.15.43:0/3236725877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3768005040 con 0x7f37741015f0 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f37737fe640 1 --2- 172.21.15.43:0/3236725877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 0x7f377410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3772ffd640 1 --2- 172.21.15.43:0/3236725877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37741015f0 0x7f377410e180 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f3768009450 tx=0x7f3768009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:50:13.606 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768007b10 con 0x7f37741015f0 2024-09-11T08:50:13.607 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f376800f040 con 0x7f37741015f0 2024-09-11T08:50:13.607 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768013bb0 con 0x7f37741015f0 2024-09-11T08:50:13.607 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376c0098b0 con 0x7f37741015f0 2024-09-11T08:50:13.607 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.604+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3774119710 con 0x7f37741015f0 2024-09-11T08:50:13.608 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.605+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37740fcd70 con 0x7f37741015f0 2024-09-11T08:50:13.608 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.606+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7f376800e930 con 0x7f37741015f0 2024-09-11T08:50:13.608 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.606+0000 7f375bfff640 1 --2- 172.21.15.43:0/3236725877 >> v2:172.21.15.43:6800/588375333 conn(0x7f375c06ef80 0x7f375c071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:50:13.608 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.607+0000 7f37737fe640 1 --2- 172.21.15.43:0/3236725877 >> v2:172.21.15.43:6800/588375333 conn(0x7f375c06ef80 0x7f375c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:50:13.608 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.607+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(774..774 src has 1..774) v4 ==== 8046+0+0 (secure 0 0 0) 0x7f376808ec00 con 0x7f37741015f0 2024-09-11T08:50:13.608 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.607+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=775}) v3 -- 0x7f375c07a9a0 con 0x7f37741015f0 2024-09-11T08:50:13.608 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.607+0000 7f37737fe640 1 --2- 172.21.15.43:0/3236725877 >> v2:172.21.15.43:6800/588375333 conn(0x7f375c06ef80 0x7f375c071430 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 crypto rx=0x7f3764009f30 tx=0x7f3764009120 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:50:13.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.611+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f376805b330 con 0x7f37741015f0 2024-09-11T08:50:13.778 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:13.779+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f377410aac0 con 0x7f37741015f0 2024-09-11T08:50:13.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:13 smithi043 ceph-mon[40592]: pgmap v1301: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:50:13.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:13 smithi043 ceph-mon[29355]: pgmap v1301: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:50:13.894 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:50:13] "GET /metrics HTTP/1.1" 200 228696 "" "Prometheus/2.43.0" 2024-09-11T08:50:13.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:13 smithi172 ceph-mon[32987]: pgmap v1301: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:50:14.556 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.556+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 7 ==== osd_map(775..775 src has 1..775) v4 ==== 288+0+0 (secure 0 0 0) 0x7f376805b070 con 0x7f37741015f0 2024-09-11T08:50:14.556 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.556+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=776}) v3 -- 0x7f375c07b2d0 con 0x7f37741015f0 2024-09-11T08:50:14.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.561+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]=0 pool '992c53e8-268f-42a0-a098-268d5016ad85' removed v775) v1 ==== 248+0+0 (secure 0 0 0) 0x7f376800eea0 con 0x7f37741015f0 2024-09-11T08:50:14.667 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.667+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3774119930 con 0x7f37741015f0 2024-09-11T08:50:14.667 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.668+0000 7f375bfff640 1 -- 172.21.15.43:0/3236725877 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]=0 pool '992c53e8-268f-42a0-a098-268d5016ad85' does not exist v775) v1 ==== 255+0+0 (secure 0 0 0) 0x7f376805acd0 con 0x7f37741015f0 2024-09-11T08:50:14.667 INFO:tasks.workunit.client.0.smithi043.stderr:pool '992c53e8-268f-42a0-a098-268d5016ad85' does not exist 2024-09-11T08:50:14.669 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 >> v2:172.21.15.43:6800/588375333 conn(0x7f375c06ef80 msgr2=0x7f375c071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:14.669 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 --2- 172.21.15.43:0/3236725877 >> v2:172.21.15.43:6800/588375333 conn(0x7f375c06ef80 0x7f375c071430 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 crypto rx=0x7f3764009f30 tx=0x7f3764009120 comp rx=0 tx=0).stop 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37741015f0 msgr2=0x7f377410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 --2- 172.21.15.43:0/3236725877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37741015f0 0x7f377410e180 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f3768009450 tx=0x7f3768009da0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 shutdown_connections 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 --2- 172.21.15.43:0/3236725877 >> v2:172.21.15.43:6800/588375333 conn(0x7f375c06ef80 0x7f375c071430 unknown :-1 s=CLOSED pgs=3497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 --2- 172.21.15.43:0/3236725877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774101f90 0x7f377410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 --2- 172.21.15.43:0/3236725877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37741015f0 0x7f377410e180 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 --2- 172.21.15.43:0/3236725877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37740fbd30 0x7f377410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 >> 172.21.15.43:0/3236725877 conn(0x7f3774072530 msgr2=0x7f37740731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 shutdown_connections 2024-09-11T08:50:14.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.670+0000 7f3779ea8640 1 -- 172.21.15.43:0/3236725877 wait complete. 2024-09-11T08:50:14.679 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool delete 741af959-956f-4579-9aef-0637359b7771 741af959-956f-4579-9aef-0637359b7771 --yes-i-really-really-mean-it 2024-09-11T08:50:14.762 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 -- 172.21.15.43:0/2733670288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c0fb360 msgr2=0x7fa46c109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:14.763 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 --2- 172.21.15.43:0/2733670288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c0fb360 0x7fa46c109530 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fa464009cd0 tx=0x7fa464004d10 comp rx=0 tx=0).stop 2024-09-11T08:50:14.763 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 -- 172.21.15.43:0/2733670288 shutdown_connections 2024-09-11T08:50:14.763 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 --2- 172.21.15.43:0/2733670288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c0fb360 0x7fa46c109530 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.763 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 --2- 172.21.15.43:0/2733670288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa46c1030d0 0x7fa46c0fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.763 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 --2- 172.21.15.43:0/2733670288 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa46c102730 0x7fa46c102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.763 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 -- 172.21.15.43:0/2733670288 >> 172.21.15.43:0/2733670288 conn(0x7fa46c074ba0 msgr2=0x7fa46c0f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:50:14.764 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 -- 172.21.15.43:0/2733670288 shutdown_connections 2024-09-11T08:50:14.764 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 -- 172.21.15.43:0/2733670288 wait complete. 2024-09-11T08:50:14.764 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 Processor -- start 2024-09-11T08:50:14.764 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.760+0000 7fa471046640 1 -- start start 2024-09-11T08:50:14.764 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa471046640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa46c0fb360 0x7fa46c112a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:50:14.765 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa471046640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c102730 0x7fa46c112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:50:14.765 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa471046640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa46c1030d0 0x7fa46c10cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:50:14.765 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa471046640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c10d240 con 0x7fa46c102730 2024-09-11T08:50:14.765 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa471046640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c10d3b0 con 0x7fa46c0fb360 2024-09-11T08:50:14.765 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa471046640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c10d570 con 0x7fa46c1030d0 2024-09-11T08:50:14.766 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa46c1030d0 0x7fa46c10cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:50:14.766 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa463fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c102730 0x7fa46c112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:50:14.766 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa46c1030d0 0x7fa46c10cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:60056/0 (socket says 172.21.15.43:60056) 2024-09-11T08:50:14.766 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 -- 172.21.15.43:0/3017535455 learned_addr learned my addr 172.21.15.43:0/3017535455 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T08:50:14.766 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46ad9d640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa46c0fb360 0x7fa46c112a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:50:14.766 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 -- 172.21.15.43:0/3017535455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa46c0fb360 msgr2=0x7fa46c112a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:14.767 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa46c0fb360 0x7fa46c112a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.767 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 -- 172.21.15.43:0/3017535455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c102730 msgr2=0x7fa46c112fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:14.767 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c102730 0x7fa46c112fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:14.767 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 -- 172.21.15.43:0/3017535455 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c005040 con 0x7fa46c1030d0 2024-09-11T08:50:14.767 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46ad9d640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa46c0fb360 0x7fa46c112a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T08:50:14.767 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa463fff640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c102730 0x7fa46c112fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:50:14.768 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa46b59e640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa46c1030d0 0x7fa46c10cbf0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7fa464009040 tx=0x7fa464004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:50:14.768 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464038070 con 0x7fa46c1030d0 2024-09-11T08:50:14.768 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4640098b0 con 0x7fa46c1030d0 2024-09-11T08:50:14.768 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fa464033e30 con 0x7fa46c1030d0 2024-09-11T08:50:14.768 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46403ca30 con 0x7fa46c1030d0 2024-09-11T08:50:14.768 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.761+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c1198b0 con 0x7fa46c1030d0 2024-09-11T08:50:14.769 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.762+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46c0fdf50 con 0x7fa46c1030d0 2024-09-11T08:50:14.769 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.763+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 21) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fa464030070 con 0x7fa46c1030d0 2024-09-11T08:50:14.770 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.763+0000 7fa4637fe640 1 --2- 172.21.15.43:0/3017535455 >> v2:172.21.15.43:6800/588375333 conn(0x7fa45806ef80 0x7fa458071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:50:14.771 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.763+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(775..775 src has 1..775) v4 ==== 7658+0+0 (secure 0 0 0) 0x7fa4640af410 con 0x7fa46c1030d0 2024-09-11T08:50:14.771 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.764+0000 7fa46ad9d640 1 --2- 172.21.15.43:0/3017535455 >> v2:172.21.15.43:6800/588375333 conn(0x7fa45806ef80 0x7fa458071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:50:14.771 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.764+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=776}) v3 -- 0x7fa45807a180 con 0x7fa46c1030d0 2024-09-11T08:50:14.771 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.764+0000 7fa46ad9d640 1 --2- 172.21.15.43:0/3017535455 >> v2:172.21.15.43:6800/588375333 conn(0x7fa45806ef80 0x7fa458071430 secure :-1 s=READY pgs=3498 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009fd0 tx=0x7fa45c009da0 comp rx=0 tx=0).ready entity=mgr.14553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:50:14.771 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.768+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa46407bc00 con 0x7fa46c1030d0 2024-09-11T08:50:14.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:14 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:14.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:14 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:14.930 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:14.931+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa46c10bbb0 con 0x7fa46c1030d0 2024-09-11T08:50:14.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:14 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.559 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.559+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(776..776 src has 1..776) v4 ==== 288+0+0 (secure 0 0 0) 0x7fa46403c3c0 con 0x7fa46c1030d0 2024-09-11T08:50:15.559 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.559+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=777}) v3 -- 0x7fa45807aa70 con 0x7fa46c1030d0 2024-09-11T08:50:15.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.564+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]=0 pool '741af959-956f-4579-9aef-0637359b7771' removed v776) v1 ==== 248+0+0 (secure 0 0 0) 0x7fa46407b5a0 con 0x7fa46c1030d0 2024-09-11T08:50:15.670 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.670+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa46c10bbb0 con 0x7fa46c1030d0 2024-09-11T08:50:15.671 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.671+0000 7fa4637fe640 1 -- 172.21.15.43:0/3017535455 <== mon.2 v2:172.21.15.43:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]=0 pool '741af959-956f-4579-9aef-0637359b7771' does not exist v776) v1 ==== 255+0+0 (secure 0 0 0) 0x7fa46407b780 con 0x7fa46c1030d0 2024-09-11T08:50:15.671 INFO:tasks.workunit.client.0.smithi043.stderr:pool '741af959-956f-4579-9aef-0637359b7771' does not exist 2024-09-11T08:50:15.673 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 >> v2:172.21.15.43:6800/588375333 conn(0x7fa45806ef80 msgr2=0x7fa458071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:15.673 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 --2- 172.21.15.43:0/3017535455 >> v2:172.21.15.43:6800/588375333 conn(0x7fa45806ef80 0x7fa458071430 secure :-1 s=READY pgs=3498 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009fd0 tx=0x7fa45c009da0 comp rx=0 tx=0).stop 2024-09-11T08:50:15.673 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa46c1030d0 msgr2=0x7fa46c10cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:50:15.673 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa46c1030d0 0x7fa46c10cbf0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7fa464009040 tx=0x7fa464004d10 comp rx=0 tx=0).stop 2024-09-11T08:50:15.673 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 shutdown_connections 2024-09-11T08:50:15.673 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 --2- 172.21.15.43:0/3017535455 >> v2:172.21.15.43:6800/588375333 conn(0x7fa45806ef80 0x7fa458071430 unknown :-1 s=CLOSED pgs=3498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:15.674 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa46c1030d0 0x7fa46c10cbf0 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:15.674 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c102730 0x7fa46c112fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:15.674 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 --2- 172.21.15.43:0/3017535455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa46c0fb360 0x7fa46c112a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:50:15.674 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.673+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 >> 172.21.15.43:0/3017535455 conn(0x7fa46c074ba0 msgr2=0x7fa46c0f9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:50:15.674 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.674+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 shutdown_connections 2024-09-11T08:50:15.674 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-11T08:50:15.674+0000 7fa471046640 1 -- 172.21.15.43:0/3017535455 wait complete. 2024-09-11T08:50:15.682 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo OK 2024-09-11T08:50:15.683 INFO:tasks.workunit.client.0.smithi043.stdout:OK 2024-09-11T08:50:15.684 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T08:50:15.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T08:50:15.731 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:50:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x[35077]: ::ffff:172.21.15.172 - - [11/Sep/2024:08:50:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:50:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:15 smithi172 ceph-mon[32987]: pgmap v1302: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-11T08:50:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:50:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:15 smithi172 ceph-mon[32987]: osdmap e775: 8 total, 8 up, 8 in 2024-09-11T08:50:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:15 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3017535455' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:15 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.752 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-11T08:50:15.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T08:50:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[29355]: pgmap v1302: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-11T08:50:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:50:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[29355]: osdmap e775: 8 total, 8 up, 8 in 2024-09-11T08:50:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3017535455' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[40592]: pgmap v1302: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-11T08:50:15.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:50:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[40592]: osdmap e775: 8 total, 8 up, 8 in 2024-09-11T08:50:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3236725877' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "992c53e8-268f-42a0-a098-268d5016ad85", "pool2": "992c53e8-268f-42a0-a098-268d5016ad85", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3017535455' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:15.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:15 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:16.074 DEBUG:teuthology.parallel:result is None 2024-09-11T08:50:16.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:50:16.100 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:50:16.101 DEBUG:teuthology.orchestra.run.smithi043:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:50:16.167 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:50:16.167 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T08:50:16.176 INFO:tasks.cephadm:Teardown begin 2024-09-11T08:50:16.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:50:16.232 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:50:16.268 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T08:50:16.268 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:50:16.288 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:50:16.328 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T08:50:16.328 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T08:50:16.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:50:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[40592]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:50:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:50:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[40592]: osdmap e776: 8 total, 8 up, 8 in 2024-09-11T08:50:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[40592]: from='client.? 172.21.15.43:0/3017535455' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:16.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[40592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 systemd[1]: Stopping Ceph mon.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[29355]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:50:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:50:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[29355]: osdmap e776: 8 total, 8 up, 8 in 2024-09-11T08:50:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[29355]: from='client.? 172.21.15.43:0/3017535455' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:16.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-mon[29355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:16 smithi172 ceph-mon[32987]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:50:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:50:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:16 smithi172 ceph-mon[32987]: osdmap e776: 8 total, 8 up, 8 in 2024-09-11T08:50:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:16 smithi172 ceph-mon[32987]: from='client.? 172.21.15.43:0/3017535455' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:16.981 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:16 smithi172 ceph-mon[32987]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "741af959-956f-4579-9aef-0637359b7771", "pool2": "741af959-956f-4579-9aef-0637359b7771", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:50:16.982 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:50:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug there is no tcmu-runner data available 2024-09-11T08:50:17.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a[29332]: 2024-09-11T08:50:16.672+0000 7f1ef0f18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:17.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a[29332]: 2024-09-11T08:50:16.672+0000 7f1ef0f18640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T08:50:17.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:16 smithi043 podman[149290]: 2024-09-11 08:50:16.723481345 +0000 UTC m=+0.202658344 container died b5c5c813fd92f986ab7255741249cf21770b7002ccd026747e659a711b55a953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:50:17.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:17 smithi043 podman[149290]: 2024-09-11 08:50:17.295934677 +0000 UTC m=+0.775111661 container cleanup b5c5c813fd92f986ab7255741249cf21770b7002ccd026747e659a711b55a953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T08:50:17.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 08:50:17 smithi043 bash[149290]: ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:50:17.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T08:50:17.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:17.789 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T08:50:17.789 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T08:50:17.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.c 2024-09-11T08:50:17.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:17 smithi043 systemd[1]: Stopping Ceph mon.c for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:18.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-c[40569]: 2024-09-11T08:50:18.177+0000 7f58a3666640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:18.568 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-c[40569]: 2024-09-11T08:50:18.177+0000 7f58a3666640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:50:18.568 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:18 smithi043 podman[149422]: 2024-09-11 08:50:18.260357814 +0000 UTC m=+0.251996925 container died d8c1e4093b5168b56504b2d0f95854b8f147c183357db35f001f5f4e79308f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:50:18.568 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:50:18] ENGINE Bus STOPPING 2024-09-11T08:50:18.568 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:50:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:50:18.568 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:50:18] ENGINE Bus STOPPED 2024-09-11T08:50:18.568 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:50:18] ENGINE Bus STARTING 2024-09-11T08:50:18.568 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:50:18] ENGINE Serving on http://:::9283 2024-09-11T08:50:18.568 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:18 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: [11/Sep/2024:08:50:18] ENGINE Bus STARTED 2024-09-11T08:50:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:18 smithi043 podman[149422]: 2024-09-11 08:50:18.693227808 +0000 UTC m=+0.684866960 container cleanup d8c1e4093b5168b56504b2d0f95854b8f147c183357db35f001f5f4e79308f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T08:50:18.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 11 08:50:18 smithi043 bash[149422]: ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-c 2024-09-11T08:50:19.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T08:50:19.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:19.165 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T08:50:19.165 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T08:50:19.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.b 2024-09-11T08:50:19.481 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:19 smithi172 systemd[1]: Stopping Ceph mon.b for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:19.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:19 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-b[32964]: 2024-09-11T08:50:19.498+0000 7f11b63c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:19.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:19 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-b[32964]: 2024-09-11T08:50:19.498+0000 7f11b63c1640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:50:19.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:19 smithi172 podman[72283]: 2024-09-11 08:50:19.632224784 +0000 UTC m=+0.270956190 container died 36ede87bebc634129e2ddb17d4b1422dbbeffe089242a0c1926567b6b44b74a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T08:50:20.230 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:20 smithi172 podman[72283]: 2024-09-11 08:50:20.122534431 +0000 UTC m=+0.761265872 container cleanup 36ede87bebc634129e2ddb17d4b1422dbbeffe089242a0c1926567b6b44b74a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T08:50:20.231 INFO:journalctl@ceph.mon.b.smithi172.stdout:Sep 11 08:50:20 smithi172 bash[72283]: ceph-327ee848-7017-11ef-bcea-c7b262605968-mon-b 2024-09-11T08:50:20.552 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T08:50:20.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:20.602 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T08:50:20.602 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T08:50:20.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:50:20.893 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:20 smithi043 systemd[1]: Stopping Ceph mgr.y for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:21.248 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:50:20.906+0000 7fa3a9fbe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:21.249 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y[29609]: 2024-09-11T08:50:20.906+0000 7fa3a9fbe640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T08:50:21.249 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:20 smithi043 podman[149567]: 2024-09-11 08:50:20.938405531 +0000 UTC m=+0.135196821 container died 9155dd89112de829e89f87389a95f70463d32238e8d5495b6101adf7ce2ba5d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:50:21.565 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:21 smithi043 podman[149567]: 2024-09-11 08:50:21.322628885 +0000 UTC m=+0.519420174 container cleanup 9155dd89112de829e89f87389a95f70463d32238e8d5495b6101adf7ce2ba5d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T08:50:21.565 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 11 08:50:21 smithi043 bash[149567]: ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-y 2024-09-11T08:50:21.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T08:50:21.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:21.739 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T08:50:21.739 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T08:50:21.739 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.x 2024-09-11T08:50:21.981 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:50:21 smithi172 systemd[1]: Stopping Ceph mgr.x for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:22.327 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:50:22 smithi172 podman[72422]: 2024-09-11 08:50:22.058365201 +0000 UTC m=+0.124573720 container died f8306e018d2c08d426fb08e674676534ef36c9062d163b8adbfb50b847ee388a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T08:50:22.694 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:50:22 smithi172 podman[72422]: 2024-09-11 08:50:22.426954327 +0000 UTC m=+0.493162846 container cleanup f8306e018d2c08d426fb08e674676534ef36c9062d163b8adbfb50b847ee388a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T08:50:22.694 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 11 08:50:22 smithi172 bash[72422]: ceph-327ee848-7017-11ef-bcea-c7b262605968-mgr-x 2024-09-11T08:50:22.859 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T08:50:22.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:22.897 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T08:50:22.897 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T08:50:22.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.0 2024-09-11T08:50:23.353 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:22 smithi043 systemd[1]: Stopping Ceph osd.0 for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:23.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:23.569Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:23.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:23.569Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:23.645 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0[45014]: 2024-09-11T08:50:23.354+0000 7f4c0af39640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:23.645 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0[45014]: 2024-09-11T08:50:23.354+0000 7f4c0af39640 -1 osd.0 776 *** Got signal Terminated *** 2024-09-11T08:50:23.645 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0[45014]: 2024-09-11T08:50:23.354+0000 7f4c0af39640 -1 osd.0 776 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:50:29.144 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:28 smithi043 podman[149697]: 2024-09-11 08:50:28.797424336 +0000 UTC m=+5.706488958 container died 707e5518d1cebd0137e890debff6eee5ffe9766974ca7ef282d55000f34ecc39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T08:50:29.511 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:29 smithi043 podman[149697]: 2024-09-11 08:50:29.269030494 +0000 UTC m=+6.178095115 container cleanup 707e5518d1cebd0137e890debff6eee5ffe9766974ca7ef282d55000f34ecc39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:50:29.511 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:29 smithi043 bash[149697]: ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0 2024-09-11T08:50:29.836 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:29 smithi043 podman[149717]: 2024-09-11 08:50:29.511907281 +0000 UTC m=+0.709798826 container remove 707e5518d1cebd0137e890debff6eee5ffe9766974ca7ef282d55000f34ecc39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:50:30.144 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:29 smithi043 podman[149812]: 2024-09-11 08:50:29.7405171 +0000 UTC m=+0.016080866 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:50:30.144 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:29 smithi043 podman[149812]: 2024-09-11 08:50:29.971062254 +0000 UTC m=+0.246626008 container create 3f053e492fb52a48054b31f34ecb15310d510b1fe7ea2fcc4f6fa4660c6f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T08:50:30.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:30 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:30.149Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:30.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:30 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:30.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:30.645 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:30 smithi043 podman[149812]: 2024-09-11 08:50:30.190206742 +0000 UTC m=+0.465770515 container init 3f053e492fb52a48054b31f34ecb15310d510b1fe7ea2fcc4f6fa4660c6f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T08:50:30.645 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:30 smithi043 podman[149812]: 2024-09-11 08:50:30.196351228 +0000 UTC m=+0.471914980 container start 3f053e492fb52a48054b31f34ecb15310d510b1fe7ea2fcc4f6fa4660c6f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:50:30.645 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:30 smithi043 podman[149812]: 2024-09-11 08:50:30.281807831 +0000 UTC m=+0.557371596 container attach 3f053e492fb52a48054b31f34ecb15310d510b1fe7ea2fcc4f6fa4660c6f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T08:50:30.645 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 08:50:30 smithi043 podman[149812]: 2024-09-11 08:50:30.370554808 +0000 UTC m=+0.646118570 container died 3f053e492fb52a48054b31f34ecb15310d510b1fe7ea2fcc4f6fa4660c6f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:50:30.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T08:50:30.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:30.995 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T08:50:30.996 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T08:50:30.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.1 2024-09-11T08:50:31.290 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:31 smithi043 systemd[1]: Stopping Ceph osd.1 for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:31.643 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:31 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:50:31.290+0000 7f5ad8e49640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:31.644 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:31 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:50:31.290+0000 7f5ad8e49640 -1 osd.1 776 *** Got signal Terminated *** 2024-09-11T08:50:31.644 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:31 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1[50928]: 2024-09-11T08:50:31.290+0000 7f5ad8e49640 -1 osd.1 776 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:50:33.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:33.569Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:33.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:33.570Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:33.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:33.570Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:36.980 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:36 smithi043 podman[149944]: 2024-09-11 08:50:36.66939731 +0000 UTC m=+5.475098389 container died 37e34d7acfd9ea50a868b01161e0e4f433d6a2705646be51dcf550b4d23e0bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T08:50:37.297 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:37 smithi043 podman[149944]: 2024-09-11 08:50:37.104967196 +0000 UTC m=+5.910668274 container cleanup 37e34d7acfd9ea50a868b01161e0e4f433d6a2705646be51dcf550b4d23e0bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:50:37.297 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:37 smithi043 bash[149944]: ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1 2024-09-11T08:50:37.643 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:37 smithi043 podman[149968]: 2024-09-11 08:50:37.298008488 +0000 UTC m=+0.623071164 container remove 37e34d7acfd9ea50a868b01161e0e4f433d6a2705646be51dcf550b4d23e0bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T08:50:38.042 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:37 smithi043 podman[150057]: 2024-09-11 08:50:37.626155349 +0000 UTC m=+0.016301264 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:50:38.042 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:37 smithi043 podman[150057]: 2024-09-11 08:50:37.740781941 +0000 UTC m=+0.130927839 container create 401fcfdb851bb23e93ab1a46095b59c3ec762378150e0036e499e97186c9f5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T08:50:38.042 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:37 smithi043 podman[150057]: 2024-09-11 08:50:37.968191339 +0000 UTC m=+0.358337246 container init 401fcfdb851bb23e93ab1a46095b59c3ec762378150e0036e499e97186c9f5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T08:50:38.042 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:37 smithi043 podman[150057]: 2024-09-11 08:50:37.973057006 +0000 UTC m=+0.363202906 container start 401fcfdb851bb23e93ab1a46095b59c3ec762378150e0036e499e97186c9f5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux ) 2024-09-11T08:50:38.042 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:38 smithi043 podman[150057]: 2024-09-11 08:50:38.043091136 +0000 UTC m=+0.433237046 container attach 401fcfdb851bb23e93ab1a46095b59c3ec762378150e0036e499e97186c9f5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T08:50:38.394 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 11 08:50:38 smithi043 podman[150057]: 2024-09-11 08:50:38.156856813 +0000 UTC m=+0.547002712 container died 401fcfdb851bb23e93ab1a46095b59c3ec762378150e0036e499e97186c9f5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T08:50:38.781 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T08:50:38.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:38.821 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T08:50:38.821 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T08:50:38.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.2 2024-09-11T08:50:39.144 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:38 smithi043 systemd[1]: Stopping Ceph osd.2 for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:39.472 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:39 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2[56903]: 2024-09-11T08:50:39.183+0000 7f5d044cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:39.473 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:39 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2[56903]: 2024-09-11T08:50:39.183+0000 7f5d044cb640 -1 osd.2 776 *** Got signal Terminated *** 2024-09-11T08:50:39.473 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:39 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2[56903]: 2024-09-11T08:50:39.183+0000 7f5d044cb640 -1 osd.2 776 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:50:40.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:40.150Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:40.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:40.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:40.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:40 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:40.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:43.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:43.570Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:43.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:43.570Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:43.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:43 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:43.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:44.816 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:44 smithi043 podman[150194]: 2024-09-11 08:50:44.516452172 +0000 UTC m=+5.446025639 container died 261c53cd213f854c571df9026f9079e9eef7623dfad9e8c35c7ddf04c2e5097c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:50:45.144 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:44 smithi043 podman[150194]: 2024-09-11 08:50:44.891288818 +0000 UTC m=+5.820862250 container cleanup 261c53cd213f854c571df9026f9079e9eef7623dfad9e8c35c7ddf04c2e5097c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T08:50:45.144 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:44 smithi043 bash[150194]: ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2 2024-09-11T08:50:45.144 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:45 smithi043 podman[150216]: 2024-09-11 08:50:45.059276102 +0000 UTC m=+0.533888933 container remove 261c53cd213f854c571df9026f9079e9eef7623dfad9e8c35c7ddf04c2e5097c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:50:45.837 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:45 smithi043 podman[150311]: 2024-09-11 08:50:45.416881824 +0000 UTC m=+0.016477505 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:50:45.837 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:45 smithi043 podman[150311]: 2024-09-11 08:50:45.526773453 +0000 UTC m=+0.126369127 container create 9fcd3dbd577a1572f34addcd9613723aad898573668bba127ed42334a4948d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T08:50:45.837 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:45 smithi043 podman[150311]: 2024-09-11 08:50:45.754255005 +0000 UTC m=+0.353850677 container init 9fcd3dbd577a1572f34addcd9613723aad898573668bba127ed42334a4948d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T08:50:45.837 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:45 smithi043 podman[150311]: 2024-09-11 08:50:45.758872018 +0000 UTC m=+0.358467692 container start 9fcd3dbd577a1572f34addcd9613723aad898573668bba127ed42334a4948d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:50:45.837 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:45 smithi043 podman[150311]: 2024-09-11 08:50:45.837529491 +0000 UTC m=+0.437125166 container attach 9fcd3dbd577a1572f34addcd9613723aad898573668bba127ed42334a4948d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T08:50:46.143 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 11 08:50:45 smithi043 podman[150311]: 2024-09-11 08:50:45.935283792 +0000 UTC m=+0.534879463 container died 9fcd3dbd577a1572f34addcd9613723aad898573668bba127ed42334a4948d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:50:46.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T08:50:46.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:46.637 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T08:50:46.637 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T08:50:46.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.3 2024-09-11T08:50:46.894 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:46 smithi043 systemd[1]: Stopping Ceph osd.3 for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:47.394 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:46 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:50:46.978+0000 7f0f7649f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:47.394 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:46 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:50:46.978+0000 7f0f7649f640 -1 osd.3 776 *** Got signal Terminated *** 2024-09-11T08:50:47.394 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:46 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:50:46.978+0000 7f0f7649f640 -1 osd.3 776 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:50:49.394 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:49 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:50:49.149+0000 7f0f742ba640 -1 osd.3 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.567639+0000 front 2024-09-11T08:50:26.567743+0000 (oldest deadline 2024-09-11T08:50:48.867744+0000) 2024-09-11T08:50:50.644 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:50 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:50:50.144+0000 7f0f742ba640 -1 osd.3 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.567639+0000 front 2024-09-11T08:50:26.567743+0000 (oldest deadline 2024-09-11T08:50:48.867744+0000) 2024-09-11T08:50:50.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:50 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:50.151Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:50.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:50 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:50.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:50.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:50 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:50.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:50.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:50 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:50.656+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:51.394 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:51 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3[63343]: 2024-09-11T08:50:51.112+0000 7f0f742ba640 -1 osd.3 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.567639+0000 front 2024-09-11T08:50:26.567743+0000 (oldest deadline 2024-09-11T08:50:48.867744+0000) 2024-09-11T08:50:51.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:51 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:51.661+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:52.778 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:52 smithi043 podman[150438]: 2024-09-11 08:50:52.502173325 +0000 UTC m=+5.632258690 container died 1898b4e8f337c6048e803233c1e5cce65f48f3c11df3a0c3607f72aa4134ee03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:50:52.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:52 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:52.658+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:52.981 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:50:52 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:50:52.654+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:50:53.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:52 smithi043 podman[150438]: 2024-09-11 08:50:52.869064808 +0000 UTC m=+5.999150217 container cleanup 1898b4e8f337c6048e803233c1e5cce65f48f3c11df3a0c3607f72aa4134ee03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T08:50:53.111 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:52 smithi043 bash[150438]: ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3 2024-09-11T08:50:53.394 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:53 smithi043 podman[150459]: 2024-09-11 08:50:53.111805832 +0000 UTC m=+0.612570578 container remove 1898b4e8f337c6048e803233c1e5cce65f48f3c11df3a0c3607f72aa4134ee03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:50:53.706 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:53 smithi043 podman[150553]: 2024-09-11 08:50:53.420985339 +0000 UTC m=+0.109033163 container create 5d7d189cde31c64a258d585032e23219805db053a25e56b781188674cf23e146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T08:50:53.706 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:53 smithi043 podman[150553]: 2024-09-11 08:50:53.328141749 +0000 UTC m=+0.016189586 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:50:53.706 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:53 smithi043 podman[150553]: 2024-09-11 08:50:53.615280884 +0000 UTC m=+0.303328709 container init 5d7d189cde31c64a258d585032e23219805db053a25e56b781188674cf23e146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T08:50:53.706 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:53 smithi043 podman[150553]: 2024-09-11 08:50:53.620514886 +0000 UTC m=+0.308562713 container start 5d7d189cde31c64a258d585032e23219805db053a25e56b781188674cf23e146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:50:53.706 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:53 smithi043 podman[150553]: 2024-09-11 08:50:53.706683455 +0000 UTC m=+0.394731292 container attach 5d7d189cde31c64a258d585032e23219805db053a25e56b781188674cf23e146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:50:53.706 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:53.570Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:53.706 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:53.570Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:50:53.707 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:50:53 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:50:53.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:50:53.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:53 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:53.643+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:53.981 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:53 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:53.868+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:50:53.981 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:50:53 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:50:53.688+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:50:54.083 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:53 smithi043 podman[150553]: 2024-09-11 08:50:53.799612249 +0000 UTC m=+0.487660083 container died 5d7d189cde31c64a258d585032e23219805db053a25e56b781188674cf23e146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T08:50:54.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T08:50:54.357 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:54 smithi043 podman[150597]: 2024-09-11 08:50:54.318472033 +0000 UTC m=+0.512979562 container remove 5d7d189cde31c64a258d585032e23219805db053a25e56b781188674cf23e146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:50:54.358 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:54 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-11T08:50:54.358 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:54 smithi043 systemd[1]: Stopped Ceph osd.3 for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:50:54.358 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 08:50:54 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.3.service: Consumed 39.715s CPU time. 2024-09-11T08:50:54.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:50:54.373 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T08:50:54.373 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T08:50:54.373 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.4 2024-09-11T08:50:54.397 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:54 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:54.136+0000 7fa6256a0640 -1 osd.4 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.772657+0000 front 2024-09-11T08:50:27.772721+0000 (oldest deadline 2024-09-11T08:50:53.672496+0000) 2024-09-11T08:50:54.660 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:54 smithi172 systemd[1]: Stopping Ceph osd.4 for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:50:54.660 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:54 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:54.607+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:54.981 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:50:54 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:50:54.660+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:50:54.981 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:54 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:54.705+0000 7fa62b08c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:50:54.981 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:54 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:54.705+0000 7fa62b08c640 -1 osd.4 776 *** Got signal Terminated *** 2024-09-11T08:50:54.981 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:54 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:54.705+0000 7fa62b08c640 -1 osd.4 776 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:50:54.981 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:54 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:54.894+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:50:55.481 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:55.120+0000 7fa6256a0640 -1 osd.4 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.772657+0000 front 2024-09-11T08:50:27.772721+0000 (oldest deadline 2024-09-11T08:50:53.672496+0000) 2024-09-11T08:50:55.922 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:55.563+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:55.923 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:50:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:50:55.682+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:50:56.231 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:56.167+0000 7fa6256a0640 -1 osd.4 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.772657+0000 front 2024-09-11T08:50:27.772721+0000 (oldest deadline 2024-09-11T08:50:53.672496+0000) 2024-09-11T08:50:56.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:55 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:55.923+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:50:56.930 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:56.585+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:56.931 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:50:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:50:56.728+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:50:57.231 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:57 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:57.124+0000 7fa6256a0640 -1 osd.4 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.772657+0000 front 2024-09-11T08:50:27.772721+0000 (oldest deadline 2024-09-11T08:50:53.672496+0000) 2024-09-11T08:50:57.232 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:56 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:56.930+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:50:57.949 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:50:57 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:50:57.695+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:50:57.949 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:57 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:57.605+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:57.949 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:57 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:57.605+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:50:58.231 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:58 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:58.132+0000 7fa6256a0640 -1 osd.4 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.772657+0000 front 2024-09-11T08:50:27.772721+0000 (oldest deadline 2024-09-11T08:50:53.672496+0000) 2024-09-11T08:50:58.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:57 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:57.949+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:50:58.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:57 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:57.949+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:50:58.911 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:50:58 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:50:58.686+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:50:58.911 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:58 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:58.593+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:58.912 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:58 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:58.593+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:50:59.231 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:59 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:59.105+0000 7fa6256a0640 -1 osd.4 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.772657+0000 front 2024-09-11T08:50:27.772721+0000 (oldest deadline 2024-09-11T08:50:53.672496+0000) 2024-09-11T08:50:59.231 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:50:59 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4[38311]: 2024-09-11T08:50:59.105+0000 7fa6256a0640 -1 osd.4 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.673151+0000 front 2024-09-11T08:50:33.673103+0000 (oldest deadline 2024-09-11T08:50:58.373030+0000) 2024-09-11T08:50:59.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:58 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:58.911+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:50:59.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:58 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:58.911+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:50:59.957 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:50:59 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:50:59.689+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:50:59.958 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:59 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:59.613+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:50:59.958 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:50:59 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:50:59.613+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:00.231 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:00 smithi172 podman[72573]: 2024-09-11 08:51:00.130678873 +0000 UTC m=+5.569193822 container died c8b435be3b5df6cf0136e3a3e23b2f51802bae5d6edbdd336ddb05ce15ccbc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:51:00.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:59 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:59.958+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:00.232 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:50:59 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:50:59.958+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:00.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:00.151Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:00.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:00.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:00.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:00 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:00.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:00.731 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:00 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:00.645+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:51:00.731 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:00 smithi172 podman[72573]: 2024-09-11 08:51:00.544466658 +0000 UTC m=+5.982981603 container cleanup c8b435be3b5df6cf0136e3a3e23b2f51802bae5d6edbdd336ddb05ce15ccbc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T08:51:00.731 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:00 smithi172 bash[72573]: ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4 2024-09-11T08:51:00.732 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:00 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:00.619+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:00.732 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:00 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:00.619+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:01.214 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:00 smithi172 podman[72593]: 2024-09-11 08:51:00.779471413 +0000 UTC m=+0.645716876 container remove c8b435be3b5df6cf0136e3a3e23b2f51802bae5d6edbdd336ddb05ce15ccbc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T08:51:01.214 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:00 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:00.916+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:01.214 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:00 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:00.916+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:01.481 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:01 smithi172 podman[72689]: 2024-09-11 08:51:01.117640215 +0000 UTC m=+0.015862867 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:51:01.481 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:01 smithi172 podman[72689]: 2024-09-11 08:51:01.221832063 +0000 UTC m=+0.120054706 container create a4b68177d538bdb18bad1d2b645eaa3c8bd1316a0cdd5d7b82f584e880d068b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T08:51:01.481 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:01 smithi172 podman[72689]: 2024-09-11 08:51:01.449397457 +0000 UTC m=+0.347620106 container init a4b68177d538bdb18bad1d2b645eaa3c8bd1316a0cdd5d7b82f584e880d068b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T08:51:01.481 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:01 smithi172 podman[72689]: 2024-09-11 08:51:01.454645428 +0000 UTC m=+0.352868081 container start a4b68177d538bdb18bad1d2b645eaa3c8bd1316a0cdd5d7b82f584e880d068b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T08:51:01.905 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:01 smithi172 podman[72689]: 2024-09-11 08:51:01.524335419 +0000 UTC m=+0.422558064 container attach a4b68177d538bdb18bad1d2b645eaa3c8bd1316a0cdd5d7b82f584e880d068b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T08:51:01.905 INFO:journalctl@ceph.osd.4.smithi172.stdout:Sep 11 08:51:01 smithi172 podman[72689]: 2024-09-11 08:51:01.629737239 +0000 UTC m=+0.527959888 container died a4b68177d538bdb18bad1d2b645eaa3c8bd1316a0cdd5d7b82f584e880d068b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T08:51:01.905 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:01 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:01.669+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:01.905 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:01 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:01.669+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:01.905 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:01 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:01.683+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:51:02.160 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:01 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:01.904+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:02.160 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:01 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:01.904+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:02.183 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T08:51:02.264 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:51:02.264 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T08:51:02.265 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T08:51:02.265 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.5 2024-09-11T08:51:02.481 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:02 smithi172 systemd[1]: Stopping Ceph osd.5 for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:02.936 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:02.714+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:02.936 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:02.714+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:02.937 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:02.607+0000 7f5dadc46640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:51:02.937 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:02.607+0000 7f5dadc46640 -1 osd.5 776 *** Got signal Terminated *** 2024-09-11T08:51:02.937 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:02.607+0000 7f5dadc46640 -1 osd.5 776 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:51:02.937 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:02.727+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:51:02.937 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:02.727+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:32.247797+0000 front 2024-09-11T08:50:32.247811+0000 (oldest deadline 2024-09-11T08:51:02.248232+0000) 2024-09-11T08:51:03.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:02.936+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:03.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:02 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:02.936+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:03.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:03.570Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:03.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:03.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:03.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:03 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:03.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:03.926 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:03 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:03.742+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:03.926 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:03 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:03.742+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:03.926 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:03 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:03.742+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:51:03.926 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:03 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:03.742+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:32.247797+0000 front 2024-09-11T08:50:32.247811+0000 (oldest deadline 2024-09-11T08:51:02.248232+0000) 2024-09-11T08:51:04.202 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:03 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:03.926+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:04.202 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:03 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:03.926+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:05.231 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:04.736+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:05.231 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:04.736+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:05.231 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:04.751+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:51:05.231 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:04.751+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:32.247797+0000 front 2024-09-11T08:50:32.247811+0000 (oldest deadline 2024-09-11T08:51:02.248232+0000) 2024-09-11T08:51:05.232 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:04.922+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:05.232 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:04 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:04.922+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:05.981 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:05.711+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:51:05.981 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:05.711+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:32.247797+0000 front 2024-09-11T08:50:32.247811+0000 (oldest deadline 2024-09-11T08:51:02.248232+0000) 2024-09-11T08:51:05.982 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:05.712+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:05.982 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:05.712+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:05.982 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:05.960+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:05.982 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:05 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:05.960+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:06.981 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:06.741+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:26.446835+0000 front 2024-09-11T08:50:26.446764+0000 (oldest deadline 2024-09-11T08:50:52.247190+0000) 2024-09-11T08:51:06.981 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5[44687]: 2024-09-11T08:51:06.741+0000 7f5dab260640 -1 osd.5 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:32.247797+0000 front 2024-09-11T08:50:32.247811+0000 (oldest deadline 2024-09-11T08:51:02.248232+0000) 2024-09-11T08:51:06.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:06.677+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:06.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:06.677+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:07.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:06.994+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:07.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:06 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:06.994+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:07.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:07 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:07.709+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:07.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:07 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:07.709+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:08.306 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:08 smithi172 podman[72822]: 2024-09-11 08:51:08.017976063 +0000 UTC m=+5.527842171 container died 6a6c7354b0ce33085e585d5aae3cedc8ebfcbeabcf6cd5a0077d60fb74bb8203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T08:51:08.306 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:08 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:08.014+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:08.306 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:08 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:08.014+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:08.638 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:08 smithi172 podman[72822]: 2024-09-11 08:51:08.413419491 +0000 UTC m=+5.923285633 container cleanup 6a6c7354b0ce33085e585d5aae3cedc8ebfcbeabcf6cd5a0077d60fb74bb8203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T08:51:08.638 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:08 smithi172 bash[72822]: ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5 2024-09-11T08:51:08.939 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:08 smithi172 podman[72843]: 2024-09-11 08:51:08.639356894 +0000 UTC m=+0.614754425 container remove 6a6c7354b0ce33085e585d5aae3cedc8ebfcbeabcf6cd5a0077d60fb74bb8203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T08:51:08.939 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:08 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:08.667+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:08.939 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:08 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:08.667+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:08.939 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:08 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:08.667+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:40.187028+0000 front 2024-09-11T08:50:40.187104+0000 (oldest deadline 2024-09-11T08:51:07.787270+0000) 2024-09-11T08:51:09.200 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:08 smithi172 podman[72937]: 2024-09-11 08:51:08.940153354 +0000 UTC m=+0.105203611 container create fe8285e6234074dfaf506718c2456c9a14c23ede2105bb3dbf9f9332fde8965d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T08:51:09.200 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:08 smithi172 podman[72937]: 2024-09-11 08:51:08.850842718 +0000 UTC m=+0.015892948 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:51:09.200 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:09 smithi172 podman[72937]: 2024-09-11 08:51:09.200829094 +0000 UTC m=+0.365879309 container init fe8285e6234074dfaf506718c2456c9a14c23ede2105bb3dbf9f9332fde8965d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:51:09.200 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:08 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:08.967+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:09.200 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:08 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:08.967+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:09.200 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:08 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:08.967+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:09.481 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:09 smithi172 podman[72937]: 2024-09-11 08:51:09.205604538 +0000 UTC m=+0.370654754 container start fe8285e6234074dfaf506718c2456c9a14c23ede2105bb3dbf9f9332fde8965d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:51:09.481 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:09 smithi172 podman[72937]: 2024-09-11 08:51:09.284170823 +0000 UTC m=+0.449221061 container attach fe8285e6234074dfaf506718c2456c9a14c23ede2105bb3dbf9f9332fde8965d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:51:09.481 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 08:51:09 smithi172 podman[72937]: 2024-09-11 08:51:09.381852342 +0000 UTC m=+0.546902568 container died fe8285e6234074dfaf506718c2456c9a14c23ede2105bb3dbf9f9332fde8965d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True) 2024-09-11T08:51:09.919 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:09 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:09.665+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:09.920 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:09 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:09.665+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:09.920 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:09 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:09.665+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:40.187028+0000 front 2024-09-11T08:50:40.187104+0000 (oldest deadline 2024-09-11T08:51:07.787270+0000) 2024-09-11T08:51:09.939 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T08:51:09.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:51:09.978 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T08:51:09.978 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T08:51:09.978 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.6 2024-09-11T08:51:10.231 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:10 smithi172 systemd[1]: Stopping Ceph osd.6 for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:10.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:10.006+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:10.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:10.006+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:10.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:10.006+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:10.493 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:10 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:10.152Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:10.494 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:10 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:10.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:10.494 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:10 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:10.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:10.666 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:10.325+0000 7f7faa071640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:51:10.667 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:10.325+0000 7f7faa071640 -1 osd.6 776 *** Got signal Terminated *** 2024-09-11T08:51:10.667 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:10.325+0000 7f7faa071640 -1 osd.6 776 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:51:10.980 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:10.666+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:10.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:10.666+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:10.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:10 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:10.666+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:40.187028+0000 front 2024-09-11T08:50:40.187104+0000 (oldest deadline 2024-09-11T08:51:07.787270+0000) 2024-09-11T08:51:11.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:11 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:11.015+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:11.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:11 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:11.015+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:11.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:11 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:11.015+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:11.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:11 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:11.692+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:11.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:11 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:11.692+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:11.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:11 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:11.692+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:40.187028+0000 front 2024-09-11T08:50:40.187104+0000 (oldest deadline 2024-09-11T08:51:07.787270+0000) 2024-09-11T08:51:12.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:12 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:12.028+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:12.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:12 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:12.028+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:12.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:12 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:12.028+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:12.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:12 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:12.660+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:12.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:12 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:12.660+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:12.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:12 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:12.660+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:40.187028+0000 front 2024-09-11T08:50:40.187104+0000 (oldest deadline 2024-09-11T08:51:07.787270+0000) 2024-09-11T08:51:13.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:13 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:13.077+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:13.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:13 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:13.077+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:13.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:13 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:13.077+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:13.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:13.572Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:13.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:13.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:13.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:13 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:13.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:13.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:13 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:13.630+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:13.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:13 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:13.630+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:13.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:13 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:13.630+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:40.187028+0000 front 2024-09-11T08:50:40.187104+0000 (oldest deadline 2024-09-11T08:51:07.787270+0000) 2024-09-11T08:51:13.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:13 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:13.630+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:47.787723+0000 front 2024-09-11T08:50:47.787736+0000 (oldest deadline 2024-09-11T08:51:13.087620+0000) 2024-09-11T08:51:14.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:14 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:14.084+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:14.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:14 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:14.084+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:14.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:14 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:14.084+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:14.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:14 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:14.085+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:14.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:14 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:14.638+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:24.385602+0000 front 2024-09-11T08:50:24.385424+0000 (oldest deadline 2024-09-11T08:50:50.285255+0000) 2024-09-11T08:51:14.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:14 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:14.638+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:33.186250+0000 front 2024-09-11T08:50:33.186160+0000 (oldest deadline 2024-09-11T08:50:56.686064+0000) 2024-09-11T08:51:14.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:14 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:14.638+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:40.187028+0000 front 2024-09-11T08:50:40.187104+0000 (oldest deadline 2024-09-11T08:51:07.787270+0000) 2024-09-11T08:51:14.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:14 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6[50995]: 2024-09-11T08:51:14.638+0000 7f7fa5e88640 -1 osd.6 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:47.787723+0000 front 2024-09-11T08:50:47.787736+0000 (oldest deadline 2024-09-11T08:51:13.087620+0000) 2024-09-11T08:51:15.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:15.065+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:15.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:15.065+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:15.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:15.065+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:15.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:15.065+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:48.386861+0000 front 2024-09-11T08:50:48.387174+0000 (oldest deadline 2024-09-11T08:51:14.286839+0000) 2024-09-11T08:51:15.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:15 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:15.065+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:15.981 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:15 smithi172 podman[73063]: 2024-09-11 08:51:15.713829366 +0000 UTC m=+5.516802119 container died ee98a54197b63ecbd0814a1eec0569a6a9b8a035846468b66fce4ab6e2510bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-09-11T08:51:16.316 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:16 smithi172 podman[73063]: 2024-09-11 08:51:16.072989912 +0000 UTC m=+5.875962707 container cleanup ee98a54197b63ecbd0814a1eec0569a6a9b8a035846468b66fce4ab6e2510bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T08:51:16.317 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:16 smithi172 bash[73063]: ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6 2024-09-11T08:51:16.317 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:16.061+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:16.317 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:16.061+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:16.317 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:16.061+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:16.317 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:16.061+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:48.386861+0000 front 2024-09-11T08:50:48.387174+0000 (oldest deadline 2024-09-11T08:51:14.286839+0000) 2024-09-11T08:51:16.317 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:16 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:16.061+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:16.626 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:16 smithi172 podman[73085]: 2024-09-11 08:51:16.317553237 +0000 UTC m=+0.598827154 container remove ee98a54197b63ecbd0814a1eec0569a6a9b8a035846468b66fce4ab6e2510bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:51:16.912 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:16 smithi172 podman[73181]: 2024-09-11 08:51:16.627644836 +0000 UTC m=+0.101872065 container create dd92d07b4ed3ced19f785fbfb90ae67530805aca19f2e762e0ffe013e3ec3e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:51:16.912 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:16 smithi172 podman[73181]: 2024-09-11 08:51:16.541638894 +0000 UTC m=+0.015866088 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:51:16.913 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:16 smithi172 podman[73181]: 2024-09-11 08:51:16.838356399 +0000 UTC m=+0.312583589 container init dd92d07b4ed3ced19f785fbfb90ae67530805aca19f2e762e0ffe013e3ec3e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T08:51:16.913 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:16 smithi172 podman[73181]: 2024-09-11 08:51:16.843278814 +0000 UTC m=+0.317505998 container start dd92d07b4ed3ced19f785fbfb90ae67530805aca19f2e762e0ffe013e3ec3e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:51:16.913 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:16 smithi172 podman[73181]: 2024-09-11 08:51:16.913159867 +0000 UTC m=+0.387387065 container attach dd92d07b4ed3ced19f785fbfb90ae67530805aca19f2e762e0ffe013e3ec3e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T08:51:17.231 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 11 08:51:17 smithi172 podman[73181]: 2024-09-11 08:51:17.017621478 +0000 UTC m=+0.491848662 container died dd92d07b4ed3ced19f785fbfb90ae67530805aca19f2e762e0ffe013e3ec3e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T08:51:17.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:17.057+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:17.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:17.057+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:17.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:17.057+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:17.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:17.057+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:48.386861+0000 front 2024-09-11T08:50:48.387174+0000 (oldest deadline 2024-09-11T08:51:14.286839+0000) 2024-09-11T08:51:17.232 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:17.057+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:17.591 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T08:51:17.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:51:17.629 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T08:51:17.629 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T08:51:17.629 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.7 2024-09-11T08:51:17.945 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 systemd[1]: Stopping Ceph osd.7 for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:18.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:17.944+0000 7f6b34275640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:51:18.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:17.944+0000 7f6b34275640 -1 osd.7 776 *** Got signal Terminated *** 2024-09-11T08:51:18.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:17 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:17.944+0000 7f6b34275640 -1 osd.7 776 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:51:18.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:18 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:18.040+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:18.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:18 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:18.040+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:18.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:18 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:18.040+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:18.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:18 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:18.040+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:48.386861+0000 front 2024-09-11T08:50:48.387174+0000 (oldest deadline 2024-09-11T08:51:14.286839+0000) 2024-09-11T08:51:18.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:18 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:18.040+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:19.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:19 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:19.056+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:19.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:19 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:19.056+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:19.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:19 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:19.056+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:19.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:19 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:19.056+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:48.386861+0000 front 2024-09-11T08:50:48.387174+0000 (oldest deadline 2024-09-11T08:51:14.286839+0000) 2024-09-11T08:51:19.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:19 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:19.056+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:20.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.019+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:20.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.019+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:20.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.019+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:20.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.019+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:48.386861+0000 front 2024-09-11T08:50:48.387174+0000 (oldest deadline 2024-09-11T08:51:14.286839+0000) 2024-09-11T08:51:20.481 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.019+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:20.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:20.153Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:20.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:20.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:20.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:20 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:20.154Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:21.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.980+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:21.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.981+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:21.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.981+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:21.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.981+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:48.386861+0000 front 2024-09-11T08:50:48.387174+0000 (oldest deadline 2024-09-11T08:51:14.286839+0000) 2024-09-11T08:51:21.231 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:20 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:20.981+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:22.460 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:21 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:21.997+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-11T08:50:27.384524+0000 front 2024-09-11T08:50:27.384606+0000 (oldest deadline 2024-09-11T08:50:53.284297+0000) 2024-09-11T08:51:22.460 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:21 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:21.997+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-11T08:50:35.585140+0000 front 2024-09-11T08:50:35.585286+0000 (oldest deadline 2024-09-11T08:50:57.284980+0000) 2024-09-11T08:51:22.461 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:21 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:21.997+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-11T08:50:43.086216+0000 front 2024-09-11T08:50:43.086603+0000 (oldest deadline 2024-09-11T08:51:08.386235+0000) 2024-09-11T08:51:22.461 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:21 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:21.997+0000 7f6b3188f640 -1 osd.7 776 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-11T08:50:48.386861+0000 front 2024-09-11T08:50:48.387174+0000 (oldest deadline 2024-09-11T08:51:14.286839+0000) 2024-09-11T08:51:22.461 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:21 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7[57106]: 2024-09-11T08:51:21.997+0000 7f6b3188f640 -1 osd.7 776 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1226 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e776) 2024-09-11T08:51:23.651 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:23 smithi172 podman[73310]: 2024-09-11 08:51:23.356665365 +0000 UTC m=+5.505912440 container died 6ea0b72a9876d18d38942bc141072e721ea168611012e56048e9115772c70d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731) 2024-09-11T08:51:23.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:23.572Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:23.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:23.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:23.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:23 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:23.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:23.981 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:23 smithi172 podman[73310]: 2024-09-11 08:51:23.734678232 +0000 UTC m=+5.883925299 container cleanup 6ea0b72a9876d18d38942bc141072e721ea168611012e56048e9115772c70d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True) 2024-09-11T08:51:23.981 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:23 smithi172 bash[73310]: ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7 2024-09-11T08:51:24.298 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:23 smithi172 podman[73345]: 2024-09-11 08:51:23.982135163 +0000 UTC m=+0.623418949 container remove 6ea0b72a9876d18d38942bc141072e721ea168611012e56048e9115772c70d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T08:51:24.585 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:24 smithi172 podman[73438]: 2024-09-11 08:51:24.299540354 +0000 UTC m=+0.110566300 container create 50ebcbf4f08850698343938d7f8317acb30ca3e079e999808485220da7e9bf03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T08:51:24.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:24 smithi172 podman[73438]: 2024-09-11 08:51:24.20510313 +0000 UTC m=+0.016129080 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:51:24.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:24 smithi172 podman[73438]: 2024-09-11 08:51:24.494546663 +0000 UTC m=+0.305572602 container init 50ebcbf4f08850698343938d7f8317acb30ca3e079e999808485220da7e9bf03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:51:24.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:24 smithi172 podman[73438]: 2024-09-11 08:51:24.499518657 +0000 UTC m=+0.310544600 container start 50ebcbf4f08850698343938d7f8317acb30ca3e079e999808485220da7e9bf03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:51:24.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:24 smithi172 podman[73438]: 2024-09-11 08:51:24.586415528 +0000 UTC m=+0.397441479 container attach 50ebcbf4f08850698343938d7f8317acb30ca3e079e999808485220da7e9bf03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T08:51:24.963 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:24 smithi172 podman[73438]: 2024-09-11 08:51:24.67754981 +0000 UTC m=+0.488575762 container died 50ebcbf4f08850698343938d7f8317acb30ca3e079e999808485220da7e9bf03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:51:25.229 INFO:journalctl@ceph.osd.7.smithi172.stdout:Sep 11 08:51:25 smithi172 podman[73481]: 2024-09-11 08:51:25.206083095 +0000 UTC m=+0.523010278 container remove 50ebcbf4f08850698343938d7f8317acb30ca3e079e999808485220da7e9bf03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T08:51:25.235 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T08:51:25.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:51:25.274 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T08:51:25.274 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T08:51:25.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T08:51:25.586 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:51:25 smithi043 systemd[1]: Stopping Ceph rgw.foo.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:25.894 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:51:25 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a[68323]: 2024-09-11T08:51:25.586+0000 7f5abcd87640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:51:25.894 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:51:25 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a[68323]: 2024-09-11T08:51:25.586+0000 7f5ac128f7c0 -1 shutting down 2024-09-11T08:51:30.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:30 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:30.153Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:30.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:30 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:30.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:30.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:30 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:30.154Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:33.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:33.573Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:33.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:33.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi043.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-11T08:51:33.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:33 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:33.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi172.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.172:8443: connect: connection refused" 2024-09-11T08:51:35.894 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:51:35 smithi043 bash[150696]: time="2024-09-11T08:51:35Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:51:35.894 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:51:35 smithi043 podman[150696]: 2024-09-11 08:51:35.607502575 +0000 UTC m=+10.131062808 container died 9564a4bb9567b7f4d9dd3b44ba4f934640c8640471c746ff52f44227a7041a22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T08:51:36.317 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:51:36 smithi043 podman[150696]: 2024-09-11 08:51:36.083751413 +0000 UTC m=+10.607311646 container cleanup 9564a4bb9567b7f4d9dd3b44ba4f934640c8640471c746ff52f44227a7041a22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T08:51:36.317 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 11 08:51:36 smithi043 bash[150696]: ceph-327ee848-7017-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T08:51:36.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T08:51:36.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:51:36.489 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T08:51:36.489 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T08:51:36.489 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-327ee848-7017-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T08:51:36.792 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 systemd[1]: Stopping Ceph prometheus.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:37.095 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.792Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a[67921]: ts=2024-09-11T08:51:36.793Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:51:37.096 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:36 smithi172 podman[73565]: 2024-09-11 08:51:36.803386187 +0000 UTC m=+0.122419055 container died 575201f09b8551343ddfe545825581d2351fa6f53345fe9a15e52b892338ee37 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:51:37.412 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:37 smithi172 podman[73565]: 2024-09-11 08:51:37.212288179 +0000 UTC m=+0.531321046 container cleanup 575201f09b8551343ddfe545825581d2351fa6f53345fe9a15e52b892338ee37 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:51:37.412 INFO:journalctl@ceph.prometheus.a.smithi172.stdout:Sep 11 08:51:37 smithi172 bash[73565]: ceph-327ee848-7017-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:51:37.567 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-327ee848-7017-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T08:51:37.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:51:37.602 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T08:51:37.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm rm-cluster --fsid 327ee848-7017-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:51:39.600 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:39 smithi043 systemd[1]: Stopping Ceph alertmanager.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:39.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:39 smithi043 ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a[77860]: ts=2024-09-11T08:51:39.600Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:51:39.894 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:39 smithi043 podman[151151]: 2024-09-11 08:51:39.641276727 +0000 UTC m=+0.148826029 container died 9c3e45c5a953ff5c13f624a10b7a80c1a41633ea24c6ba0aff7c7ba289b37900 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:51:40.356 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:40 smithi043 podman[151151]: 2024-09-11 08:51:40.105484711 +0000 UTC m=+0.613034013 container cleanup 9c3e45c5a953ff5c13f624a10b7a80c1a41633ea24c6ba0aff7c7ba289b37900 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:51:40.356 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:40 smithi043 bash[151151]: ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:51:40.643 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:40 smithi043 podman[151167]: 2024-09-11 08:51:40.356815852 +0000 UTC m=+0.713121124 container remove 9c3e45c5a953ff5c13f624a10b7a80c1a41633ea24c6ba0aff7c7ba289b37900 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:51:40.644 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:40 smithi043 podman[151167]: 2024-09-11 08:51:40.499319976 +0000 UTC m=+0.855625251 volume remove 38df3fcc48ed56aa9a3f700af7bcf8b081e94ba697fadee421730d31019b743c 2024-09-11T08:51:41.139 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:40 smithi043 systemd[1]: Stopping Ceph node-exporter.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:41.139 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:40 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:51:41.139 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:40 smithi043 systemd[1]: Stopped Ceph alertmanager.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:51:41.139 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 11 08:51:40 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.946s CPU time. 2024-09-11T08:51:41.393 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:41 smithi043 podman[151279]: 2024-09-11 08:51:41.14001046 +0000 UTC m=+0.118765316 container died 97777e91719481730cdcea91cdc4aca1fb6bf2ed7ed8d04d24fc7950e1c009cc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:51:41.785 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:41 smithi043 podman[151279]: 2024-09-11 08:51:41.501845034 +0000 UTC m=+0.480599885 container cleanup 97777e91719481730cdcea91cdc4aca1fb6bf2ed7ed8d04d24fc7950e1c009cc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:51:41.785 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:41 smithi043 bash[151279]: ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T08:51:42.079 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:41 smithi043 podman[151296]: 2024-09-11 08:51:41.786406583 +0000 UTC m=+0.647980790 container remove 97777e91719481730cdcea91cdc4aca1fb6bf2ed7ed8d04d24fc7950e1c009cc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:51:42.079 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:41 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:51:42.079 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:41 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T08:51:42.079 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:41 smithi043 systemd[1]: Stopped Ceph node-exporter.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:51:42.079 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 11 08:51:41 smithi043 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 3.903s CPU time. 2024-09-11T08:51:42.481 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:42 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a[66409]: logger=cleanup t=2024-09-11T08:51:42.158987367Z level=info msg="Completed cleanup jobs" duration=11.224473ms 2024-09-11T08:51:42.606 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm rm-cluster --fsid 327ee848-7017-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:51:44.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:43 smithi172 systemd[1]: Stopping Ceph iscsi.iscsi.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:44.231 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:44 smithi172 ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a[62549]: debug Shutdown received 2024-09-11T08:51:54.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:54 smithi172 bash[73948]: time="2024-09-11T08:51:54Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:51:54.475 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:54 smithi172 podman[73948]: 2024-09-11 08:51:54.184540807 +0000 UTC m=+10.101131380 container died e679348ce9fda312d97877b59b332ae7f6afa7c03e70cbc47a922a704e7b3d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:51:54.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:54 smithi172 podman[73948]: 2024-09-11 08:51:54.599171496 +0000 UTC m=+10.515762066 container cleanup e679348ce9fda312d97877b59b332ae7f6afa7c03e70cbc47a922a704e7b3d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T08:51:54.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:54 smithi172 bash[73948]: ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T08:51:55.114 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:54 smithi172 podman[73973]: 2024-09-11 08:51:54.775179218 +0000 UTC m=+0.587291469 container remove e679348ce9fda312d97877b59b332ae7f6afa7c03e70cbc47a922a704e7b3d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T08:51:55.114 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:54 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:51:55.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 podman[74020]: 2024-09-11 08:51:55.11532583 +0000 UTC m=+0.100960413 container died 2eb59dd0efa815b0799f4f1e381aeb922dd229703c7b5121a44512863d482143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T08:51:55.711 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 podman[74020]: 2024-09-11 08:51:55.485471899 +0000 UTC m=+0.471106473 container cleanup 2eb59dd0efa815b0799f4f1e381aeb922dd229703c7b5121a44512863d482143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T08:51:55.712 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 bash[74020]: ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T08:51:55.712 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T08:51:55.712 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 62415 (conmon) remains running after unit stopped. 2024-09-11T08:51:55.712 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 74037 (podman) remains running after unit stopped. 2024-09-11T08:51:55.712 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 systemd[1]: Stopped Ceph iscsi.iscsi.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:51:55.712 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 3.234s CPU time. 2024-09-11T08:51:55.981 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:55 smithi172 systemd[1]: Stopping Ceph grafana.a for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:55.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi172.stdout:Sep 11 08:51:55 smithi172 podman[74037]: 2024-09-11 08:51:55.712534612 +0000 UTC m=+0.600381664 container remove 2eb59dd0efa815b0799f4f1e381aeb922dd229703c7b5121a44512863d482143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T08:51:56.431 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:56 smithi172 podman[74162]: 2024-09-11 08:51:56.15323257 +0000 UTC m=+0.124135294 container died fe61939015f31635142f6cf4d93c76afb402d8e3bb8a318c00e6b6cc1597b220 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, architecture=x86_64, io.openshift.expose-services=, version=8.5, io.buildah.version=1.32.0, vcs-type=git, release=239.1651231664, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner ) 2024-09-11T08:51:56.731 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:56 smithi172 podman[74162]: 2024-09-11 08:51:56.515143795 +0000 UTC m=+0.486046624 container cleanup fe61939015f31635142f6cf4d93c76afb402d8e3bb8a318c00e6b6cc1597b220 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.expose-services=, vendor=Red Hat, Inc., version=8.5, vcs-type=git, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, architecture=x86_64, maintainer=Paul Cuzner , description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-09-11T08:51:56.731 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:56 smithi172 bash[74162]: ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a 2024-09-11T08:51:57.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:56 smithi172 podman[74180]: 2024-09-11 08:51:56.808408207 +0000 UTC m=+0.657463662 container remove fe61939015f31635142f6cf4d93c76afb402d8e3bb8a318c00e6b6cc1597b220 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-grafana-a, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, io.openshift.expose-services=, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, maintainer=Paul Cuzner , name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, description=Ceph Grafana Container, version=8.5, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public) 2024-09-11T08:51:57.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:56 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:51:57.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:56 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T08:51:57.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:56 smithi172 systemd[1]: Stopped Ceph grafana.a for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:51:57.119 INFO:journalctl@ceph.grafana.a.smithi172.stdout:Sep 11 08:51:56 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@grafana.a.service: Consumed 11.445s CPU time. 2024-09-11T08:51:57.398 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:57 smithi172 systemd[1]: Stopping Ceph node-exporter.b for 327ee848-7017-11ef-bcea-c7b262605968... 2024-09-11T08:51:57.678 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:57 smithi172 podman[74302]: 2024-09-11 08:51:57.399026199 +0000 UTC m=+0.115240126 container died 7e793bfb3ce6b09033cdca599ea82604f46fb778ffa1c246367b28d9fee05555 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:51:57.981 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:57 smithi172 podman[74302]: 2024-09-11 08:51:57.777773994 +0000 UTC m=+0.493987913 container cleanup 7e793bfb3ce6b09033cdca599ea82604f46fb778ffa1c246367b28d9fee05555 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:51:57.981 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:57 smithi172 bash[74302]: ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T08:51:58.269 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:57 smithi172 podman[74319]: 2024-09-11 08:51:57.998935919 +0000 UTC m=+0.600526424 container remove 7e793bfb3ce6b09033cdca599ea82604f46fb778ffa1c246367b28d9fee05555 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-327ee848-7017-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:51:58.269 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:58 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:51:58.269 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:58 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T08:51:58.269 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:58 smithi172 systemd[1]: Stopped Ceph node-exporter.b for 327ee848-7017-11ef-bcea-c7b262605968. 2024-09-11T08:51:58.269 INFO:journalctl@ceph.node-exporter.b.smithi172.stdout:Sep 11 08:51:58 smithi172 systemd[1]: ceph-327ee848-7017-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 3.613s CPU time. 2024-09-11T08:51:58.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:51:58.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:51:58.830 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T08:51:58.832 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280/remote/smithi043/crash 2024-09-11T08:51:58.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:51:58.893 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:51:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:51:58.895 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280/remote/smithi172/crash 2024-09-11T08:51:58.896 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:51:58.929 INFO:teuthology.orchestra.run.smithi172.stderr:tar: /var/lib/ceph/327ee848-7017-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:51:58.929 INFO:teuthology.orchestra.run.smithi172.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:51:58.930 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T08:51:58.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-09-11T08:51:58.961 INFO:teuthology.orchestra.run.smithi043.stderr:grep: /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T08:51:58.963 INFO:tasks.cephadm:Compressing logs... 2024-09-11T08:51:58.963 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:51:59.006 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:51:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T08:51:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:: No such file or directory 2024-09-11T08:51:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T08:51:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:51:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T08:51:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T08:51:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T08:51:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T08:51:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:: No such file or directory 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/tcmu-runner.log: /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.7.log: 63.9% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:51:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:51:59.037 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T08:51:59.037 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mgr.y.log: /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T08:51:59.043 INFO:teuthology.orchestra.run.smithi172.stderr: 92.1% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T08:51:59.043 INFO:teuthology.orchestra.run.smithi172.stderr: 90.9% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:51:59.047 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:51:59.047 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:51:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 93.0% 93.8% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T08:51:59.078 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:51:59.454 INFO:teuthology.orchestra.run.smithi043.stderr: 90.8% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T08:52:00.069 INFO:teuthology.orchestra.run.smithi043.stderr: 92.3% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T08:52:00.382 INFO:teuthology.orchestra.run.smithi172.stderr: 92.9% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T08:52:01.536 INFO:teuthology.orchestra.run.smithi043.stderr: 92.0% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T08:52:04.628 INFO:teuthology.orchestra.run.smithi172.stderr: 94.0% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T08:52:04.675 INFO:teuthology.orchestra.run.smithi043.stderr: 93.9% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T08:52:04.687 INFO:teuthology.orchestra.run.smithi172.stderr: 94.0% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T08:52:04.698 INFO:teuthology.orchestra.run.smithi172.stderr: 93.9% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T08:52:05.048 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T08:52:05.214 INFO:teuthology.orchestra.run.smithi172.stderr: 94.1% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T08:52:05.215 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-09-11T08:52:05.215 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m6.195s 2024-09-11T08:52:05.216 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m23.566s 2024-09-11T08:52:05.216 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m0.887s 2024-09-11T08:52:05.787 INFO:teuthology.orchestra.run.smithi043.stderr: 94.1% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T08:52:05.874 INFO:teuthology.orchestra.run.smithi043.stderr: 94.1% -- replaced with /var/log/ceph/327ee848-7017-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T08:52:05.876 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-11T08:52:05.876 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m6.856s 2024-09-11T08:52:05.876 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m28.288s 2024-09-11T08:52:05.877 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m0.945s 2024-09-11T08:52:05.877 INFO:tasks.cephadm:Archiving logs... 2024-09-11T08:52:05.877 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280/remote/smithi043/log 2024-09-11T08:52:05.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:52:09.696 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280/remote/smithi172/log 2024-09-11T08:52:09.697 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:52:12.931 INFO:tasks.cephadm:Removing cluster... 2024-09-11T08:52:12.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm rm-cluster --fsid 327ee848-7017-11ef-bcea-c7b262605968 --force 2024-09-11T08:52:13.455 DEBUG:teuthology.orchestra.run.smithi172:> sudo cephadm rm-cluster --fsid 327ee848-7017-11ef-bcea-c7b262605968 --force 2024-09-11T08:52:13.956 INFO:tasks.cephadm:Teardown complete 2024-09-11T08:52:13.957 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T08:52:13.966 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T08:52:13.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:52:13.969 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:52:14.040 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T08:52:14.071 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T08:52:14.071 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:52:14.075 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:52:14.088 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:52:14.090 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:52:14.092 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:52:14.093 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-11T08:52:14.093 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -2226us[-2378us] +/- 41ms 2024-09-11T08:52:14.093 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 +1820us[+1668us] +/- 51ms 2024-09-11T08:52:14.093 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +1112us[ +960us] +/- 69ms 2024-09-11T08:52:14.093 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:52:14.095 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:52:14.095 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-11T08:52:14.095 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 -2223us[-2444us] +/- 41ms 2024-09-11T08:52:14.095 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 +1808us[+1588us] +/- 51ms 2024-09-11T08:52:14.096 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +1118us[+1118us] +/- 69ms 2024-09-11T08:52:14.096 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:52:14.096 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T08:52:14.363 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T08:52:14.364 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T08:52:14.406 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:52:14.487 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:52:14.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:52:14.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:52:14.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.619 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T08:52:14.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T08:52:14.635 INFO:teuthology.task.internal:Duration was 2060.363636 seconds 2024-09-11T08:52:14.636 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T08:52:14.644 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T08:52:14.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:52:14.646 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:52:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:52:14.697 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:52:15.017 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T08:52:15.018 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T08:52:15.018 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:52:15.039 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-09-11T08:52:15.039 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:52:15.080 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T08:52:15.080 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:52:15.083 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:52:15.146 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T08:52:15.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:52:15.150 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:52:15.592 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T08:52:15.601 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T08:52:15.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:52:15.627 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:52:15.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T08:52:15.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:52:15.670 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:52:15.691 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-11T08:52:15.715 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-09-11T08:52:15.747 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:52:15.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.762 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:52:15.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.802 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T08:52:15.811 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T08:52:15.812 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280/remote/smithi043 2024-09-11T08:52:15.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:52:15.854 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901280/remote/smithi172 2024-09-11T08:52:15.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:52:15.891 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T08:52:15.891 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:52:15.894 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:52:15.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T08:52:15.951 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T08:52:15.951 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T08:52:15.959 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T08:52:15.959 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:52:15.962 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:52:15.975 INFO:teuthology.orchestra.run.smithi043.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:52 /home/ubuntu/cephtest 2024-09-11T08:52:15.995 INFO:teuthology.orchestra.run.smithi172.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:52 /home/ubuntu/cephtest 2024-09-11T08:52:15.996 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T08:52:16.006 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T08:52:16.026 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} duration: 2060.363636493683 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-11T08:52:16.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:52:16.104 INFO:teuthology.run:pass